Logan CusanoandClaude Opus 5
ee633cbe46
Edge nodes are deployed to arbitrary locations by arbitrary people, so the
broker has to be reachable from the internet and secured on its own merits
rather than by a VPN.
Three defects made that impossible. The broker only had a plaintext 1883
listener; every node shared one drb-node password; and the ACL pattern used
%c, the client-supplied client id, so any holder of that shared password
could set client_id to another node and take over its namespace. The comment
claiming this cryptographically prevented cross-node access was wrong and is
gone.
Authentication now uses mosquitto 2.x's built-in dynamic-security plugin on
the stock eclipse-mosquitto image. c2-core administers it over the control
topic, creating each node's client on approval with username=<node_id> and
password=<its node_keys api_key>, attached to a role whose ACL is nodes/%u/#
against the authenticated username. One credential, one revocation point.
An HTTP-callback plugin was implemented first and rejected: that project is
archived upstream, which is not an acceptable dependency on an
internet-facing broker.
Because dynsec state is a second source of truth alongside Firestore,
approve/reissue/delete now write to the broker first and surface a 502
rather than drifting, and c2-core reconciles every approved node into dynsec
on startup.
Adds node self-enrollment (POST /nodes/enroll, GET /nodes/{id}/credentials)
so a new node can obtain its key over HTTPS without an operator handling
secrets by hand. Enrolling an already-approved node_id is refused on the
fleet token alone — otherwise a leaked token plus a guessable id would let
an attacker steal a live node's key before the real node asked for it.
Pickup secrets are stored hashed and returned once, and the endpoint is rate
limited per source IP.
Infrastructure: an 8883 TLS listener fed by Caddy's certificate via a
systemd path unit, a firewall rule for it, and Caddy now 404s /internal/*
so the api vhost cannot proxy internal routes.
Also fixes CORS, which allowed https://app.<domain> while the frontend is
served on the bare domain — every call from the portal would have failed —
and widens the vault gitignore to a glob, since ansible-vault leaves
backup siblings that the exact-name rule left committable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
162 lines
4.9 KiB
YAML
162 lines
4.9 KiB
YAML
---
|
|
# First-time setup: clone repo, write secrets, pull pre-built images and start stack.
|
|
# Images are built and pushed by Gitea CI — this role never builds on the VM.
|
|
|
|
# update: true (was false) — with update disabled, every re-run of this playbook
|
|
# redeployed the code that happened to be on the VM at first clone, so any fix
|
|
# pushed to main was invisible here and the only way to ship one was CI or a
|
|
# manual pull. force: true discards local edits made on the VM; the templated
|
|
# .env files and Caddyfile live outside git tracking, so nothing generated by
|
|
# this role is at risk.
|
|
- name: Clone or update repo
|
|
git:
|
|
repo: "{{ repo_url }}"
|
|
dest: "{{ app_dir }}"
|
|
version: main
|
|
update: true
|
|
force: true
|
|
become: false
|
|
|
|
- name: Set ownership of app directory
|
|
file:
|
|
path: "{{ app_dir }}"
|
|
state: directory
|
|
owner: "{{ ssh_user }}"
|
|
group: "{{ ssh_user }}"
|
|
recurse: true
|
|
|
|
- name: Template top-level .env (docker-compose MQTT creds + registry)
|
|
template:
|
|
src: root.env.j2
|
|
dest: "{{ app_dir }}/.env"
|
|
owner: "{{ ssh_user }}"
|
|
group: "{{ ssh_user }}"
|
|
mode: "0600"
|
|
|
|
- name: Template c2-core .env
|
|
template:
|
|
src: c2-core.env.j2
|
|
dest: "{{ app_dir }}/drb-c2-core/.env"
|
|
owner: "{{ ssh_user }}"
|
|
group: "{{ ssh_user }}"
|
|
mode: "0600"
|
|
|
|
- name: Template discord-bot .env
|
|
template:
|
|
src: discord-bot.env.j2
|
|
dest: "{{ app_dir }}/drb-server-discord-bot/.env"
|
|
owner: "{{ ssh_user }}"
|
|
group: "{{ ssh_user }}"
|
|
mode: "0600"
|
|
|
|
- name: Template frontend .env
|
|
template:
|
|
src: frontend.env.j2
|
|
dest: "{{ app_dir }}/drb-frontend/.env"
|
|
owner: "{{ ssh_user }}"
|
|
group: "{{ ssh_user }}"
|
|
mode: "0600"
|
|
|
|
- name: Deploy Caddyfile
|
|
template:
|
|
src: Caddyfile.j2
|
|
dest: /etc/caddy/Caddyfile
|
|
owner: root
|
|
group: root
|
|
mode: "0644"
|
|
notify: Reload Caddy
|
|
|
|
# --- MQTT TLS cert sync (Caddy -> mosquitto) --------------------------------
|
|
# See MQTT-PUBLIC-AUTH-PLAN.md "Infra". mosquitto reads its cert from this
|
|
# directory (docker-compose.prod.yml bind-mounts it in); nothing but root can
|
|
# read Caddy's own cert storage, so a systemd path unit + oneshot service
|
|
# copies a readable copy out and SIGHUPs the broker on every change.
|
|
|
|
- name: Create mosquitto certs directory
|
|
file:
|
|
path: /opt/drb/mosquitto-certs
|
|
state: directory
|
|
owner: root
|
|
group: root
|
|
mode: "0700"
|
|
|
|
# dynamic-security.json (node credentials — see app/internal/dynsec.py)
|
|
# lives here. Root-owned is fine: the mosquitto container itself runs as
|
|
# root (no `user` directive in mosquitto.conf, matching the pre-existing
|
|
# setup this project already ran before the dynsec change), so it can
|
|
# read/write this directory directly without any host-side chown dance.
|
|
- name: Create mosquitto data directory
|
|
file:
|
|
path: /opt/drb/mosquitto-data
|
|
state: directory
|
|
owner: root
|
|
group: root
|
|
mode: "0700"
|
|
|
|
- name: Deploy MQTT cert-sync script
|
|
template:
|
|
src: sync-mqtt-cert.sh.j2
|
|
dest: /opt/drb/sync-mqtt-cert.sh
|
|
owner: root
|
|
group: root
|
|
mode: "0700"
|
|
|
|
- name: Deploy MQTT cert-sync systemd service unit
|
|
template:
|
|
src: mqtt-cert-sync.service.j2
|
|
dest: /etc/systemd/system/mqtt-cert-sync.service
|
|
owner: root
|
|
group: root
|
|
mode: "0644"
|
|
notify: Reload systemd daemon
|
|
|
|
- name: Deploy MQTT cert-sync systemd path unit
|
|
template:
|
|
src: mqtt-cert-sync.path.j2
|
|
dest: /etc/systemd/system/mqtt-cert-sync.path
|
|
owner: root
|
|
group: root
|
|
mode: "0644"
|
|
notify: Reload systemd daemon
|
|
|
|
# Flush the daemon-reload handler now (rather than at end-of-play) so the
|
|
# path unit is registered and actively watching BEFORE the "Reload Caddy"
|
|
# handler below fires and Caddy goes to obtain the mqtt.{{ domain }} cert —
|
|
# otherwise the unit could miss the very first PathChanged event.
|
|
- name: Apply pending handlers (systemd daemon-reload)
|
|
meta: flush_handlers
|
|
|
|
- name: Enable and start MQTT cert-sync path unit
|
|
ansible.builtin.systemd_service:
|
|
name: mqtt-cert-sync.path
|
|
state: started
|
|
enabled: true
|
|
|
|
# Best-effort initial sync in case Caddy already has a cert from a previous
|
|
# run (e.g. re-running this playbook after the first successful deploy) —
|
|
# the path unit only fires on a CHANGE, so it won't pick up a cert that was
|
|
# already sitting there unchanged before it started watching. Non-fatal if
|
|
# nothing exists yet (first-ever run, before Caddy has issued anything).
|
|
- name: Best-effort initial MQTT cert sync
|
|
command: /opt/drb/sync-mqtt-cert.sh
|
|
register: _initial_sync
|
|
changed_when: "'copied cert' in _initial_sync.stdout"
|
|
failed_when: false
|
|
|
|
- name: Log in to container registry
|
|
command: >
|
|
docker login {{ vault_registry_host }}
|
|
-u {{ vault_registry_user }}
|
|
-p {{ vault_registry_token }}
|
|
no_log: true
|
|
|
|
- name: Pull pre-built images and start stack
|
|
community.docker.docker_compose_v2:
|
|
project_src: "{{ app_dir }}"
|
|
files:
|
|
- docker-compose.yml
|
|
- docker-compose.prod.yml
|
|
pull: always
|
|
build: never
|
|
state: present
|