Logan CusanoandClaude Opus 5
ee633cbe46
Edge nodes are deployed to arbitrary locations by arbitrary people, so the
broker has to be reachable from the internet and secured on its own merits
rather than by a VPN.
Three defects made that impossible. The broker only had a plaintext 1883
listener; every node shared one drb-node password; and the ACL pattern used
%c, the client-supplied client id, so any holder of that shared password
could set client_id to another node and take over its namespace. The comment
claiming this cryptographically prevented cross-node access was wrong and is
gone.
Authentication now uses mosquitto 2.x's built-in dynamic-security plugin on
the stock eclipse-mosquitto image. c2-core administers it over the control
topic, creating each node's client on approval with username=<node_id> and
password=<its node_keys api_key>, attached to a role whose ACL is nodes/%u/#
against the authenticated username. One credential, one revocation point.
An HTTP-callback plugin was implemented first and rejected: that project is
archived upstream, which is not an acceptable dependency on an
internet-facing broker.
Because dynsec state is a second source of truth alongside Firestore,
approve/reissue/delete now write to the broker first and surface a 502
rather than drifting, and c2-core reconciles every approved node into dynsec
on startup.
Adds node self-enrollment (POST /nodes/enroll, GET /nodes/{id}/credentials)
so a new node can obtain its key over HTTPS without an operator handling
secrets by hand. Enrolling an already-approved node_id is refused on the
fleet token alone — otherwise a leaked token plus a guessable id would let
an attacker steal a live node's key before the real node asked for it.
Pickup secrets are stored hashed and returned once, and the endpoint is rate
limited per source IP.
Infrastructure: an 8883 TLS listener fed by Caddy's certificate via a
systemd path unit, a firewall rule for it, and Caddy now 404s /internal/*
so the api vhost cannot proxy internal routes.
Also fixes CORS, which allowed https://app.<domain> while the frontend is
served on the bare domain — every call from the portal would have failed —
and widens the vault gitignore to a glob, since ansible-vault leaves
backup siblings that the exact-name rule left committable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
52 lines
2.3 KiB
Django/Jinja
52 lines
2.3 KiB
Django/Jinja
#!/bin/bash
|
|
# Managed by Ansible — do not edit manually.
|
|
#
|
|
# Copies Caddy's managed TLS cert for mqtt.{{ domain }} out of Caddy's
|
|
# storage (root:caddy, 0700 — nothing else can read it) into a location the
|
|
# mosquitto container can read, then SIGHUPs the broker so it picks up the
|
|
# new cert without a full restart.
|
|
#
|
|
# Triggered by mqtt-cert-sync.path.j2 (a systemd path unit) watching the
|
|
# source cert file for changes — a path unit rather than cron so this fires
|
|
# on the actual write instead of racing a polling interval.
|
|
#
|
|
# UNVERIFIED: the exact source path below assumes Caddy's default file
|
|
# storage layout and Let's Encrypt's production ACME directory name. This
|
|
# has not been confirmed against a real Caddy cert issuance for this
|
|
# project — check `caddy storage` / find the actual path under
|
|
# /var/lib/caddy the first time this runs, and correct CADDY_CERT_DIR below
|
|
# if it doesn't match.
|
|
#
|
|
# UNVERIFIED: mosquitto 2.x reloading TLS certs on SIGHUP without dropping
|
|
# connections is documented upstream but untested here. If listener 8883
|
|
# doesn't pick up the new cert (check `docker compose logs mosquitto` after
|
|
# a sync), replace the `kill -s HUP` line below with a full
|
|
# `docker compose ... restart mosquitto` instead.
|
|
set -euo pipefail
|
|
|
|
DOMAIN="mqtt.{{ domain }}"
|
|
CADDY_CERT_DIR="/var/lib/caddy/.local/share/caddy/certificates/acme-v02.api.letsencrypt.org-directory/${DOMAIN}"
|
|
DEST_DIR="/opt/drb/mosquitto-certs"
|
|
APP_DIR="{{ app_dir }}"
|
|
|
|
SRC_CERT="${CADDY_CERT_DIR}/${DOMAIN}.crt"
|
|
SRC_KEY="${CADDY_CERT_DIR}/${DOMAIN}.key"
|
|
|
|
if [ ! -f "$SRC_CERT" ] || [ ! -f "$SRC_KEY" ]; then
|
|
echo "sync-mqtt-cert: source cert/key not found yet at $CADDY_CERT_DIR — Caddy may not have issued it yet." >&2
|
|
exit 0
|
|
fi
|
|
|
|
mkdir -p "$DEST_DIR"
|
|
# Copy, don't symlink — nothing outside the caddy user can read the
|
|
# originals (0700-owned), so mosquitto (running as a different container/
|
|
# user) needs its own readable copy, not a pointer to an unreadable file.
|
|
cp "$SRC_CERT" "$DEST_DIR/mqtt.crt"
|
|
cp "$SRC_KEY" "$DEST_DIR/mqtt.key"
|
|
chmod 600 "$DEST_DIR/mqtt.crt" "$DEST_DIR/mqtt.key"
|
|
chown root:root "$DEST_DIR/mqtt.crt" "$DEST_DIR/mqtt.key"
|
|
|
|
cd "$APP_DIR"
|
|
docker compose -f docker-compose.yml -f docker-compose.prod.yml kill -s HUP mosquitto
|
|
echo "sync-mqtt-cert: copied cert for ${DOMAIN} and sent SIGHUP to mosquitto."
|