Logan CusanoandClaude Opus 5
ee633cbe46
Edge nodes are deployed to arbitrary locations by arbitrary people, so the
broker has to be reachable from the internet and secured on its own merits
rather than by a VPN.
Three defects made that impossible. The broker only had a plaintext 1883
listener; every node shared one drb-node password; and the ACL pattern used
%c, the client-supplied client id, so any holder of that shared password
could set client_id to another node and take over its namespace. The comment
claiming this cryptographically prevented cross-node access was wrong and is
gone.
Authentication now uses mosquitto 2.x's built-in dynamic-security plugin on
the stock eclipse-mosquitto image. c2-core administers it over the control
topic, creating each node's client on approval with username=<node_id> and
password=<its node_keys api_key>, attached to a role whose ACL is nodes/%u/#
against the authenticated username. One credential, one revocation point.
An HTTP-callback plugin was implemented first and rejected: that project is
archived upstream, which is not an acceptable dependency on an
internet-facing broker.
Because dynsec state is a second source of truth alongside Firestore,
approve/reissue/delete now write to the broker first and surface a 502
rather than drifting, and c2-core reconciles every approved node into dynsec
on startup.
Adds node self-enrollment (POST /nodes/enroll, GET /nodes/{id}/credentials)
so a new node can obtain its key over HTTPS without an operator handling
secrets by hand. Enrolling an already-approved node_id is refused on the
fleet token alone — otherwise a leaked token plus a guessable id would let
an attacker steal a live node's key before the real node asked for it.
Pickup secrets are stored hashed and returned once, and the endpoint is rate
limited per source IP.
Infrastructure: an 8883 TLS listener fed by Caddy's certificate via a
systemd path unit, a firewall rule for it, and Caddy now 404s /internal/*
so the api vhost cannot proxy internal routes.
Also fixes CORS, which allowed https://app.<domain> while the frontend is
served on the bare domain — every call from the portal would have failed —
and widens the vault gitignore to a glob, since ansible-vault leaves
backup siblings that the exact-name rule left committable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
50 lines
2.4 KiB
YAML
50 lines
2.4 KiB
YAML
# Production overrides — used on the VM.
|
|
# Run with: docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
|
#
|
|
# Differences from dev:
|
|
# - MQTT port 1883 is NOT published to the host (stays on the Docker bridge).
|
|
# Edge nodes reach it via WireGuard tunnel to the Docker bridge IP.
|
|
# - c2-core and frontend ports are only bound to localhost (Caddy proxies them).
|
|
# - restart: always (instead of unless-stopped) for hard reboots.
|
|
|
|
services:
|
|
# ports AND volumes both need !override here, not !reset/a plain list —
|
|
# compose merges list-type fields by APPENDING across -f files. A plain
|
|
# list (or !reset on volumes) would leave dev's mosquitto_certs named
|
|
# volume mounted at /mosquitto/certs alongside this bind mount, and two
|
|
# mounts targeting the same path is exactly the "address already in use"-
|
|
# style footgun the c2-core override below already hit once with ports.
|
|
# mosquitto-data is now a host bind mount too (not just certs) — it holds
|
|
# dynamic-security.json, the broker's only record of node credentials
|
|
# (see app/internal/dynsec.py "TWO-SOURCES-OF-TRUTH"). A named Docker
|
|
# volume already survives normal redeploys (git pull && compose pull &&
|
|
# up -d never passes -v), but the bind mount makes it inspectable/
|
|
# backupable the same way the cert directory already is. NOT read-only —
|
|
# mosquitto writes dynamic-security.json here.
|
|
mosquitto:
|
|
restart: always
|
|
ports: !override
|
|
- "8883:8883" # TLS only, published. 1883 stays internal (docker bridge, c2-core's own login).
|
|
volumes: !override
|
|
- ./drb-c2-core/mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf:ro
|
|
- /opt/drb/mosquitto-data:/mosquitto/data
|
|
- /opt/drb/mosquitto-certs:/mosquitto/certs:ro # fed by the cert-sync systemd unit, see infra/ansible
|
|
|
|
# !override, not a plain list: compose MERGES `ports` by appending, so a plain
|
|
# list leaves the base file's "8888:8000" in place alongside this one. The
|
|
# container then tries to bind 8888 twice — 0.0.0.0 and 127.0.0.1 — and the
|
|
# second bind fails with "address already in use". It also silently defeated
|
|
# the whole point of this override, publishing the port on every interface.
|
|
c2-core:
|
|
restart: always
|
|
ports: !override
|
|
- "127.0.0.1:8888:8000" # Caddy proxies, not exposed publicly
|
|
|
|
discord-bot:
|
|
restart: always
|
|
|
|
frontend:
|
|
restart: always
|
|
ports: !override
|
|
- "127.0.0.1:3000:3000" # Caddy proxies, not exposed publicly
|