Files
server-26/infra/ansible/roles/deploy/tasks/main.yml
T
Logan CusanoandClaude Opus 5 55cd1110df Give mosquitto's bind-mounted dirs to uid 1883, not root
The broker crash-looped on every deploy: "Unable to load server certificate
/mosquitto/certs/mqtt.crt ... Permission denied". The cert-sync script wrote
600 root:root into a 0700 root:root directory, on the assumption that
mosquitto runs as root inside its container. It does not — the stock
eclipse-mosquitto entrypoint drops privileges to the in-image mosquitto
user, confirmed on the server as uid=1883(mosquitto) gid=1883(mosquitto),
and the broker's own log says so on every start.

Certs dir is now root:1883 0750 with the cert 0644 and the key 0640, and
the data dir is 1883:1883 recursively — recursively because mosquitto
WRITES dynamic-security.json there, and a root-owned file left by an
earlier deploy would still be unwritable after a directory-only chown.

Also drops the "unverified Caddy cert path" note: a real issuance confirmed
the path, producing CN=mqtt.drb.cusano.net signed by Let's Encrypt.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-16 13:29:26 -04:00

172 lines
5.6 KiB
YAML

---
# First-time setup: clone repo, write secrets, pull pre-built images and start stack.
# Images are built and pushed by Gitea CI — this role never builds on the VM.
# update: true (was false) — with update disabled, every re-run of this playbook
# redeployed the code that happened to be on the VM at first clone, so any fix
# pushed to main was invisible here and the only way to ship one was CI or a
# manual pull. force: true discards local edits made on the VM; the templated
# .env files and Caddyfile live outside git tracking, so nothing generated by
# this role is at risk.
- name: Clone or update repo
git:
repo: "{{ repo_url }}"
dest: "{{ app_dir }}"
version: main
update: true
force: true
become: false
- name: Set ownership of app directory
file:
path: "{{ app_dir }}"
state: directory
owner: "{{ ssh_user }}"
group: "{{ ssh_user }}"
recurse: true
- name: Template top-level .env (docker-compose MQTT creds + registry)
template:
src: root.env.j2
dest: "{{ app_dir }}/.env"
owner: "{{ ssh_user }}"
group: "{{ ssh_user }}"
mode: "0600"
- name: Template c2-core .env
template:
src: c2-core.env.j2
dest: "{{ app_dir }}/drb-c2-core/.env"
owner: "{{ ssh_user }}"
group: "{{ ssh_user }}"
mode: "0600"
- name: Template discord-bot .env
template:
src: discord-bot.env.j2
dest: "{{ app_dir }}/drb-server-discord-bot/.env"
owner: "{{ ssh_user }}"
group: "{{ ssh_user }}"
mode: "0600"
- name: Template frontend .env
template:
src: frontend.env.j2
dest: "{{ app_dir }}/drb-frontend/.env"
owner: "{{ ssh_user }}"
group: "{{ ssh_user }}"
mode: "0600"
- name: Deploy Caddyfile
template:
src: Caddyfile.j2
dest: /etc/caddy/Caddyfile
owner: root
group: root
mode: "0644"
notify: Reload Caddy
# --- MQTT TLS cert sync (Caddy -> mosquitto) --------------------------------
# See MQTT-PUBLIC-AUTH-PLAN.md "Infra". mosquitto reads its cert from this
# directory (docker-compose.prod.yml bind-mounts it in); nothing but root can
# read Caddy's own cert storage, so a systemd path unit + oneshot service
# copies a readable copy out and SIGHUPs the broker on every change.
# root:1883 0750, not root:root 0700. The stock eclipse-mosquitto entrypoint
# drops privileges to the in-image `mosquitto` user (uid/gid 1883), so a
# root-only directory makes the broker fail to read its own cert and
# crash-loop: "Unable to load server certificate ... Permission denied".
# The host has no `mosquitto` user, hence the numeric gid.
- name: Create mosquitto certs directory
file:
path: /opt/drb/mosquitto-certs
state: directory
owner: root
group: "1883"
mode: "0750"
# dynamic-security.json (node credentials — see app/internal/dynsec.py)
# lives here, and mosquitto WRITES it, so this must be owned by the uid the
# broker actually runs as (1883), not root. The earlier assumption that the
# container runs as root was wrong — the image's entrypoint drops privileges
# to the `mosquitto` user, which a real deploy proved by failing to read a
# root-owned cert. Same numeric-gid reasoning as the certs directory above.
- name: Create mosquitto data directory
file:
path: /opt/drb/mosquitto-data
state: directory
owner: "1883"
group: "1883"
mode: "0700"
# recurse so an existing root-owned dynamic-security.json / mosquitto.db
# left behind by the earlier root-owned deploy gets fixed too — chowning
# only the directory would leave the broker unable to rewrite them.
recurse: true
- name: Deploy MQTT cert-sync script
template:
src: sync-mqtt-cert.sh.j2
dest: /opt/drb/sync-mqtt-cert.sh
owner: root
group: root
mode: "0700"
- name: Deploy MQTT cert-sync systemd service unit
template:
src: mqtt-cert-sync.service.j2
dest: /etc/systemd/system/mqtt-cert-sync.service
owner: root
group: root
mode: "0644"
notify: Reload systemd daemon
- name: Deploy MQTT cert-sync systemd path unit
template:
src: mqtt-cert-sync.path.j2
dest: /etc/systemd/system/mqtt-cert-sync.path
owner: root
group: root
mode: "0644"
notify: Reload systemd daemon
# Flush the daemon-reload handler now (rather than at end-of-play) so the
# path unit is registered and actively watching BEFORE the "Reload Caddy"
# handler below fires and Caddy goes to obtain the mqtt.{{ domain }} cert —
# otherwise the unit could miss the very first PathChanged event.
- name: Apply pending handlers (systemd daemon-reload)
meta: flush_handlers
- name: Enable and start MQTT cert-sync path unit
ansible.builtin.systemd_service:
name: mqtt-cert-sync.path
state: started
enabled: true
# Best-effort initial sync in case Caddy already has a cert from a previous
# run (e.g. re-running this playbook after the first successful deploy) —
# the path unit only fires on a CHANGE, so it won't pick up a cert that was
# already sitting there unchanged before it started watching. Non-fatal if
# nothing exists yet (first-ever run, before Caddy has issued anything).
- name: Best-effort initial MQTT cert sync
command: /opt/drb/sync-mqtt-cert.sh
register: _initial_sync
changed_when: "'copied cert' in _initial_sync.stdout"
failed_when: false
- name: Log in to container registry
command: >
docker login {{ vault_registry_host }}
-u {{ vault_registry_user }}
-p {{ vault_registry_token }}
no_log: true
- name: Pull pre-built images and start stack
community.docker.docker_compose_v2:
project_src: "{{ app_dir }}"
files:
- docker-compose.yml
- docker-compose.prod.yml
pull: always
build: never
state: present