The broker crash-looped on every deploy: "Unable to load server certificate
/mosquitto/certs/mqtt.crt ... Permission denied". The cert-sync script wrote
600 root:root into a 0700 root:root directory, on the assumption that
mosquitto runs as root inside its container. It does not — the stock
eclipse-mosquitto entrypoint drops privileges to the in-image mosquitto
user, confirmed on the server as uid=1883(mosquitto) gid=1883(mosquitto),
and the broker's own log says so on every start.
Certs dir is now root:1883 0750 with the cert 0644 and the key 0640, and
the data dir is 1883:1883 recursively — recursively because mosquitto
WRITES dynamic-security.json there, and a root-owned file left by an
earlier deploy would still be unwritable after a directory-only chown.
Also drops the "unverified Caddy cert path" note: a real issuance confirmed
the path, producing CN=mqtt.drb.cusano.net signed by Let's Encrypt.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Shallow clones were never a Gitea packing bug. An intruder had set
uploadpack.packObjectsHook in Gitea's HOME gitconfig, pointing at a
non-executable dropper, so every upload-pack died mid-pack. That hook is
gone and --depth=1 clones are verified working, so fetch-depth: 0 buys
nothing but slower CI. See INCIDENT-2026-08-11.md.
infra/Caddyfile was dead: ansible templates Caddyfile.j2 to
/etc/caddy/Caddyfile, and nothing ever deployed the static copy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
actions/checkout defaults to depth=1, and Gitea aborted generating that pack
with a bad pack header protocol error on all three retries, failing the build
before any image was pushed. A full fetch avoids the shallow-pack path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
app_url advertised https://app.<domain>, which has never had a DNS record —
the frontend is served on the bare domain by Caddy. Adds mqtt_host so the
broker endpoint nodes connect to is discoverable from terraform output.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Edge nodes are deployed to arbitrary locations by arbitrary people, so the
broker has to be reachable from the internet and secured on its own merits
rather than by a VPN.
Three defects made that impossible. The broker only had a plaintext 1883
listener; every node shared one drb-node password; and the ACL pattern used
%c, the client-supplied client id, so any holder of that shared password
could set client_id to another node and take over its namespace. The comment
claiming this cryptographically prevented cross-node access was wrong and is
gone.
Authentication now uses mosquitto 2.x's built-in dynamic-security plugin on
the stock eclipse-mosquitto image. c2-core administers it over the control
topic, creating each node's client on approval with username=<node_id> and
password=<its node_keys api_key>, attached to a role whose ACL is nodes/%u/#
against the authenticated username. One credential, one revocation point.
An HTTP-callback plugin was implemented first and rejected: that project is
archived upstream, which is not an acceptable dependency on an
internet-facing broker.
Because dynsec state is a second source of truth alongside Firestore,
approve/reissue/delete now write to the broker first and surface a 502
rather than drifting, and c2-core reconciles every approved node into dynsec
on startup.
Adds node self-enrollment (POST /nodes/enroll, GET /nodes/{id}/credentials)
so a new node can obtain its key over HTTPS without an operator handling
secrets by hand. Enrolling an already-approved node_id is refused on the
fleet token alone — otherwise a leaked token plus a guessable id would let
an attacker steal a live node's key before the real node asked for it.
Pickup secrets are stored hashed and returned once, and the endpoint is rate
limited per source IP.
Infrastructure: an 8883 TLS listener fed by Caddy's certificate via a
systemd path unit, a firewall rule for it, and Caddy now 404s /internal/*
so the api vhost cannot proxy internal routes.
Also fixes CORS, which allowed https://app.<domain> while the frontend is
served on the bare domain — every call from the portal would have failed —
and widens the vault gitignore to a glob, since ansible-vault leaves
backup siblings that the exact-name rule left committable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Only drb.cusano.net and api.drb.cusano.net have public A records, so the
app.<domain> vhost had no cert to present and the bare domain — the record
that actually exists — matched no site at all, producing
ERR_SSL_PROTOCOL_ERROR in the browser.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A password containing "$fP" was interpolated away by compose, giving
mosquitto and c2-core two different passwords and producing
"MQTT connect refused: Not authorized" with nothing in the logs pointing at
the cause. Recorded next to the values so the next person generating
credentials sees it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Compose interpolates the top-level .env, so a password containing "$fP" was
read as the variable $fP and replaced with an empty string — hence the
repeated "The \"fP\" variable is not set" warnings on every compose command.
The env_file templates are not interpolated, so c2-core kept the literal
password while mosquitto's entrypoint received the mangled one. The two sides
disagreed and c2-core could not authenticate to the broker. Escaping $ as $$
here (and only here) makes compose collapse it back to the real value.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
docker-compose.prod.yml: compose merges `ports` by appending, so the prod
override left the base file's 8888:8000 and 3000:3000 in place next to the
127.0.0.1-scoped ones. Each container tried to bind its port twice and the
second bind failed with "address already in use", so c2-core and frontend
could never start. It also meant the localhost-only binding never applied —
both ports were published on every interface. Marked both `!override`, the
same way mosquitto already used `!reset`.
infra/ansible:
- add the missing "Reload Caddy" handler; the Deploy Caddyfile task notified
a handler that did not exist, which aborts the play
- guard mkswap/swapon on whether /swapfile is already active, so a second run
does not fail on "mounted" / "Device or resource busy"
- git task now updates instead of clone-once, otherwise a re-run redeploys
whatever code was on the VM at first clone
- vault.yml.example: correct the registry token comment to read-only scope
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Terraform: e2-micro VM (us-east1-b, free tier), static IP, SSH/web
firewall rules, IAM bindings for Firestore + GCS; imports existing
drb-calls bucket and c2-server Firestore database into state
- Gitea CI: build c2-core, discord-bot, frontend images and push to
git.vpn.cusano.net registry; SSH deploy pulls pre-built images (no
build on VM)
- Ansible: first-time setup only — git clone, env files from vault,
Caddyfile, docker login + compose pull + up; no rsync or on-VM builds
- docker-compose: add image: ${REGISTRY}/name:latest alongside build:
so local dev and CI registry both work
- gitignore: add Terraform state, lock, tfvars, ansible secrets
Provisions e2-micro VM (us-east1-b, free tier) with static IP, SSH and
web firewall rules, Docker + Caddy startup script, and IAM bindings for
Firestore and GCS access via ADC. Imports existing drb-calls bucket and
c2-server Firestore database into state. Ansible roles handle first-time
setup (swap, docker group) and all subsequent deploys via rsync + docker
compose, with secrets managed via Ansible Vault. DNS stays on AWS Route 53.
Avatar (initials) + display name, email, admin badge
Account section: email, UID, role, join date, last sign-in
Discord section: link status with username/user ID/linked date, or the get-code flow if unlinked, plus unlink button
Sign out button at the bottom
Trip-level tags: admins configure available tags in the trip header (inline add/remove pills). The AI can also create new tags via the add_tag tool.
Event tags: selectable in the Add Event modal, shown as colored pills on event cards in the timeline, and on AI suggestion cards.
AI integration: sees available tags in its system prompt, applies them when proposing events, can create new ones with add_tag.
Discord: tags shown as inline code blocks under each event in /trip view.
Colors: auto-assigned from an 8-color palette by tag index, consistent everywhere.
Focus: textarea gets refocused via inputRef after the AI response (or error) lands
Persistence: chat history saved to localStorage keyed by trip ID, loaded on mount — survives refreshes
Switch from legacy Places textsearch and Directions APIs (disabled on
this project) to Places API (New) and Routes API (New). Both places.py
and the assistant's _places_search helper updated. Also fixes uid()
recursive self-call in trips page and adds Places API response logging.
auth.py
secrets.compare_digest replaces == for service key comparison (timing-safe)
Added require_service_key — bot-only endpoints (trip/event join/leave)
Added require_service_key_or_admin — node commands/config (bot via service key OR dashboard admin via Firebase)
Added _RateLimiter with three shared instances: trip_chat_limiter (20/5min per user), summarize_limiter (5/10min per incident), bootstrap_limiter (2/hr per system)
nodes.py
send_command and assign_system now require require_service_key_or_admin — the Discord bot can still call them via service key, but regular Firebase users are blocked
tokens.py
add_token, flush_tokens, set_preferred_system, delete_token all require require_admin_token
Token masking changed from token[:10] + "…" + token[-4:] to "•••" + token[-4:]
systems.py
All write endpoints (create, update, delete, ai-flags, ten-codes, vocabulary writes, bootstrap) now require require_admin_token
bootstrap_vocabulary also calls bootstrap_limiter.check(system_id)
incidents.py
POST /incidents/summarize (bulk) now requires require_admin_token
POST /incidents/{id}/summarize now calls summarize_limiter.check(incident_id)
trips.py
join_trip, leave_trip, join_event, leave_event require require_service_key — only the Discord bot can set Discord attendee identity
delete_trip, delete_event require require_service_key_or_admin
trip_chat rate-limited per caller UID, history stripped to user/assistant roles only, user message truncated to 2000 chars, Maps query strings capped at 200 chars
upload.py
Rejects files larger than settings.upload_max_bytes (default 100MB) with 413
storage.py
_safe_audio_filename() derives GCS object name from call_id + allowlisted extension, completely ignoring the client-supplied filename
config.py
Added upload_max_bytes: int = 100 * 1024 * 1024
Both Dockerfiles — python:3.14-slim → python:3.12-slim
lib/types.ts — TripRecord and TripEvent types
lib/c2api.ts — getTrips, getTrip, createTrip, deleteTrip, createTripEvent, deleteTripEvent
lib/useTrips.ts — Firestore realtime hook on the trips collection, ordered by start date
app/trips/page.tsx — List page split into Upcoming / Past sections, card click navigates to detail, "+ New Trip" modal for admins with all fields including date range and maps link
app/trips/[id]/page.tsx — Detail page fetched via C2 API (gets trip + events in one call), day-by-day itinerary with time, location, maps link, notes, and Discord attendees. Add Event modal (date constrained to trip range). Admin-only delete trip + remove event.
components/Nav.tsx — Trips link added to the nav
New /trips router with full CRUD, attendee management, and nested
events. Events validate date is within parent trip range and inherit
trip location when not explicitly set. Leaving a trip cascades
removal from all its events.
New TripCommands cog with /trip create, list, view, delete, join,
leave and /trip event add, remove, join, leave. Event autocomplete
is scoped to the selected trip. Enforces must-be-on-trip rule for
event joins with a clear error message.
- Create path (line ~1274): title only uses "at {location}" when location_coords is also set
- Update path (line ~1226): same guard — best_coords must be truthy alongside best_location
Frontend (MapView.tsx):
- Desktop sidebar: cards with location_coords → <button> fly-to; cards without → <a href> that navigates to the incident page with "View details →" text
- Mobile drawer: same split — with coords fly-to+close, without coords navigate via <a>
- Removed the "no coords" italic placeholder text; the card behavior itself makes it clear
app/map/page.tsx
Removed IncidentCard component and the incidents grid below the map — the on-map sidebar inside MapView is the single display
Moved kiosk exit button from top-3 left-3 (overlapping zoom controls) to bottom-[5.5rem] left-3
components/MapView.tsx
Fixed popup "View incident →" link — adds stopPropagation() + window.location.href to prevent Leaflet intercepting the click
Added "View details →" link on each sidebar incident card so you can navigate from the map panel without opening a popup
Added "News Alerts" overlay layer (placeholder, ready for RSS/feed integration)
lib/types.ts
Added preferred_token_id?: string | null to SystemRecord
lib/c2api.ts
Added setPreferredToken(tokenId, systemId) calling PUT /tokens/{tokenId}/prefer/{systemId} (backend already existed)
app/systems/page.tsx
Added PreferredTokenPanel component — loads the token pool lazily on expand, shows radio buttons to set/clear the preferred token, displayed on each system card above the AI flags panel
- correlator: unit_overlap on dispatch channels now applies content
divergence check when the call has geocoded coords but the incident
doesn't; previously this gap caused unrelated calls to merge into
stale incidents (e.g. patrol officer at a second scene 70 min later)
- STT: switch default model from gpt-4o-transcribe to whisper-1, which
faithfully transcribes all exchanges in multi-PTT recordings; gpt-4o
was silently dropping utterances, starving the correlation engine
- STT: remove vocabulary from the Whisper prompt; whisper-1 echoes
prompted terms into noise/silence, skewing extracted incident data;
vocabulary context is now applied exclusively in the GPT extraction
step (build_gpt_vocab_block) where it is used as reference only
When the induction loop proposes a new vocabulary term, it now records
which sampled call(s) most likely produced the suggestion. Admins see
a collapsible "▶ source" player under each pending term showing the
audio clip and transcript, so they can hear what was actually said
before approving or dismissing.
- vocabulary_learner: track sampled call docs, attach source_call_ids
to each pending term via word-overlap search with fallback
- types: VocabularyPendingTerm.source_call_ids?: string[]
- c2api: add getCall(id) using existing GET /calls/{call_id} endpoint
- VocabularyPanel: SourceCallPlayer component — lazy-loads call on
first expand, shows audio controls + transcript snippet
The loop slept 24h before its first pass, so suggestions would never
appear unless the server was up for a full day. Move the sleep to the
end so the first induction pass runs ~30s after startup.
Logan CusanoandClaude Opus 5
logan