"Update Node" is not an OTA update — it only restarts the container, and only reaches nodes already on MQTT #15

New Issue

2026-10-04T00:56:15-04:00

logan commented

2026-10-04 00:56:15 -04:00

Found 2026-10-04 while diagnosing Toughny (node-26#14).

The frontend's Update Node button (Server/drb-frontend/app/nodes/[id]/page.tsx:308-314) sends MQTT command node_update. The node handler (drb-edge-node/app/main.py:172-183) is an explicit TODO: it publishes offline and os._exit(0) so restart: unless-stopped brings the same container back. No docker compose pull, no git pull, no new image unless someone already pulled one on the host.

Two problems:

Mislabelled: the UI implies an update; it is a restart. An operator pushing a fix fleet-wide gets nothing.
Unreachable when it matters most: delivered over MQTT, so a node that can't authenticate (Toughny, #14) or is offline never receives it. There is no out-of-band channel.

Fix direction (per the TODO): host-level updater (systemd unit / timer) that pulls images and recreates the stack, triggered by MQTT and by a periodic HTTPS poll of C2 so a node locked out of the broker can still self-heal. Pair with #14's HTTPS key pickup.

Found 2026-10-04 while diagnosing Toughny (node-26#14). The frontend's **Update Node** button (`Server/drb-frontend/app/nodes/[id]/page.tsx:308-314`) sends MQTT command `node_update`. The node handler (`drb-edge-node/app/main.py:172-183`) is an explicit TODO: it publishes offline and `os._exit(0)` so `restart: unless-stopped` brings the same container back. No `docker compose pull`, no git pull, no new image unless someone already pulled one on the host. Two problems: 1. **Mislabelled:** the UI implies an update; it is a restart. An operator pushing a fix fleet-wide gets nothing. 2. **Unreachable when it matters most:** delivered over MQTT, so a node that can't authenticate (Toughny, #14) or is offline never receives it. There is no out-of-band channel. Fix direction (per the TODO): host-level updater (systemd unit / timer) that pulls images and recreates the stack, triggered by MQTT *and* by a periodic HTTPS poll of C2 so a node locked out of the broker can still self-heal. Pair with #14's HTTPS key pickup.

logan referenced this issue from a commit

2026-10-04 01:10:49 -04:00

Node self-heals its API key; one-stop install.sh; real host-level OTA updater (#14, #15)

logan referenced this issue

2026-10-04 01:11:23 -04:00

Node self-heals its API key; one-stop install.sh; real host-level OTA updater (#14, #15) #16

logan referenced this issue

2026-10-04 01:11:46 -04:00

OTA updater trusts the git host and registry with no signature or digest check — a compromised Gitea is root on every node #17

logan commented

2026-10-04 01:12:07 -04:00

Fix is up and not merged: node-26#16 (host updater) + server-26#194 (endpoint and button).

node_update writes configs/update-requested. The drb-update.path unit sees it and runs drb-update.service.
drb-update.timer (~15 min) POSTs /nodes/{id}/update-check over HTTPS, authenticated with the api_key or the pickup secret. Offline and locked-out nodes still get the update this way.
The updater pulls only the repo, ref and registry in root-owned /etc/drb/updater.conf. C2's reply is a timestamp only. It aborts if the git origin or registry doesn't match.
The admin button is now "Update Node Software". It sets update_requested_at and also sends MQTT.

Nodes installed before this need install.sh re-run once to get the updater. Release integrity (unsigned tags/images) is tracked in #17.

Fix is up and **not merged**: node-26#16 (host updater) + server-26#194 (endpoint and button). - `node_update` writes `configs/update-requested`. The `drb-update.path` unit sees it and runs `drb-update.service`. - `drb-update.timer` (~15 min) POSTs `/nodes/{id}/update-check` over HTTPS, authenticated with the api_key or the pickup secret. Offline and locked-out nodes still get the update this way. - The updater pulls only the repo, ref and registry in root-owned `/etc/drb/updater.conf`. C2's reply is a timestamp only. It aborts if the git origin or registry doesn't match. - The admin button is now "Update Node Software". It sets `update_requested_at` and also sends MQTT. Nodes installed before this need install.sh re-run once to get the updater. Release integrity (unsigned tags/images) is tracked in #17.

logan referenced this issue from a commit

2026-10-04 01:26:05 -04:00

Merge pull request 'Node self-heals its API key; one-stop install.sh; real host-level OTA updater (#14, #15)' (#16) from fix/14-15-provision-ota into main

logan commented

2026-10-04 01:26:42 -04:00

Fixed: node-26#16 (f46c798) + server-26#194 (ae0ece9); v1 re-cut at 1e1dab6. Toughny itself still needs the one-time SSH re-run (steps above) — existing nodes can only receive the updater by running the new install.sh once.

Fixed: node-26#16 (f46c798) + server-26#194 (ae0ece9); v1 re-cut at 1e1dab6. Toughny itself still needs the one-time SSH re-run (steps above) — existing nodes can only receive the updater by running the new install.sh once.

logan closed this issue

2026-10-04 01:26:42 -04:00

logan added the area/node-ota

type

bug

labels 2026-10-05 02:05:09 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: logan/node-26#15