Logan CusanoandClaude Opus 5
ee633cbe46
Edge nodes are deployed to arbitrary locations by arbitrary people, so the
broker has to be reachable from the internet and secured on its own merits
rather than by a VPN.
Three defects made that impossible. The broker only had a plaintext 1883
listener; every node shared one drb-node password; and the ACL pattern used
%c, the client-supplied client id, so any holder of that shared password
could set client_id to another node and take over its namespace. The comment
claiming this cryptographically prevented cross-node access was wrong and is
gone.
Authentication now uses mosquitto 2.x's built-in dynamic-security plugin on
the stock eclipse-mosquitto image. c2-core administers it over the control
topic, creating each node's client on approval with username=<node_id> and
password=<its node_keys api_key>, attached to a role whose ACL is nodes/%u/#
against the authenticated username. One credential, one revocation point.
An HTTP-callback plugin was implemented first and rejected: that project is
archived upstream, which is not an acceptable dependency on an
internet-facing broker.
Because dynsec state is a second source of truth alongside Firestore,
approve/reissue/delete now write to the broker first and surface a 502
rather than drifting, and c2-core reconciles every approved node into dynsec
on startup.
Adds node self-enrollment (POST /nodes/enroll, GET /nodes/{id}/credentials)
so a new node can obtain its key over HTTPS without an operator handling
secrets by hand. Enrolling an already-approved node_id is refused on the
fleet token alone — otherwise a leaked token plus a guessable id would let
an attacker steal a live node's key before the real node asked for it.
Pickup secrets are stored hashed and returned once, and the endpoint is rate
limited per source IP.
Infrastructure: an 8883 TLS listener fed by Caddy's certificate via a
systemd path unit, a firewall rule for it, and Caddy now 404s /internal/*
so the api vhost cannot proxy internal routes.
Also fixes CORS, which allowed https://app.<domain> while the frontend is
served on the bare domain — every call from the portal would have failed —
and widens the vault gitignore to a glob, since ansible-vault leaves
backup siblings that the exact-name rule left committable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
48 lines
2.2 KiB
Plaintext
48 lines
2.2 KiB
Plaintext
# Auth: mosquitto's own built-in dynamic-security plugin — NOT
|
|
# mosquitto-go-auth (that project is archived upstream, no CVE patches;
|
|
# rejected for an internet-facing broker). This plugin ships in and is
|
|
# maintained alongside the official eclipse-mosquitto image itself.
|
|
# See MQTT-PUBLIC-AUTH-PLAN.md and app/internal/dynsec.py for the full
|
|
# design (bootstrap, roles, the two-sources-of-truth reconcile).
|
|
#
|
|
# Plugin path is DERIVED FROM SOURCE (docker/2.1-alpine/Dockerfile in
|
|
# eclipse-mosquitto/mosquitto), not observed by running the image —
|
|
# nothing in this project executes/pulls images from this machine. Verify
|
|
# it on first real deploy: `docker compose logs mosquitto` will say
|
|
# "Error: Unable to load plugin" at the exact path below if it's wrong for
|
|
# whatever patch tag ends up pinned.
|
|
plugin /usr/lib/mosquitto_dynamic_security.so
|
|
# Lives on the same persistent volume as `persistence_location` below —
|
|
# one durable volume for all broker state, survives redeploys.
|
|
plugin_opt_config_file /mosquitto/data/dynamic-security.json
|
|
|
|
allow_anonymous false
|
|
# No password_file/acl_file directive anywhere in this file — the plugin
|
|
# above is the only registered auth backend. There is no "coexist" mode:
|
|
# nothing else is registered to conflict with it.
|
|
|
|
# Internal, plaintext — c2-core's own connection only (its dynsec-admin
|
|
# control-plane calls AND its regular data-plane pub/sub both use this).
|
|
# Never published to the host in prod (docker-compose.prod.yml removes the
|
|
# port mapping); external nodes use the TLS listener below instead.
|
|
listener 1883
|
|
|
|
# Public, TLS — edge nodes connect here as username=node_id, password=api_key
|
|
# (the same credential /upload already trusts via node_keys), authorized by
|
|
# the "node" dynsec role (nodes/%u/# — %u is the dynsec-authenticated
|
|
# username, fixing the old %c-based ACL's client-ID-spoofing hole). Cert/key
|
|
# come from infra/ansible's Caddy cert-sync unit; see
|
|
# MQTT-PUBLIC-AUTH-PLAN.md "Infra" and the "Rollout order" cert-verification
|
|
# step for what happens before that cert exists.
|
|
listener 8883
|
|
certfile /mosquitto/certs/mqtt.crt
|
|
keyfile /mosquitto/certs/mqtt.key
|
|
|
|
persistence true
|
|
persistence_location /mosquitto/data/
|
|
|
|
log_dest stdout
|
|
log_type error
|
|
log_type warning
|
|
log_type notice
|