Logan CusanoandClaude Opus 5
33a247d306
The 2026-08-20 production dump had 4 of 6 sampled incidents as junk chains,
the worst being f5190670: 68 calls over 4h09m, 44 units, 12 tags, at least
13 genuinely distinct events. 58 of 133 linked calls took the fast/thin
path, which is the one path that attaches a call with no fit test at all.
Three defects combined to produce that, and all three are fixed here.
1. What counted as thin was wrong.
is_thin_call was "not units and not vehicles and not coords". A real
dispatch qualified as thin whenever no unit ID parsed and the geocode
failed - six of them did in that dump, including "All units head over to
the powerhouse, 55 Hyman Hills Road ... she's 87 years old", a brand new
job that attached to the four-hour chain and then overwrote its location
and its title. A call is now substantive if it carries tags, a location
string, a severity above routine, or is a reassignment; only genuinely
content-free housekeeping ("10-4", "Copy") stays thin. Those calls now go
through _call_fits_incident like everything else, which on a dispatch
backbone with no positive signal means they open their own incident or
orphan rather than merging.
The reassignment clause closes a self-defeating guard: upload.py blanks
units when dispatch pulls a unit onto a NEW job, specifically to stop
unit-overlap chaining - and blanking units made the call thin, routing it
to the only path with no fit check. The guard produced the merge it
existed to prevent.
2. The thin path was bounded on dispatch channels only.
Every other talkgroup fell through to "thin_pool = tg_recent": any
incident idle up to tg_fast_path_idle_minutes (90), no single-candidate
requirement, no fit test. The 30-second tier-1 / single-candidate tier-2
structure now applies to all channels. Non-dispatch gets its own window,
TG_THIN_IDLE_MINUTES=15, rather than sharing the dispatch value: a
tactical channel really is dedicated to one scene so it earns longer, but
15 sits inside the 20-minute tactical-default window already used in
_call_fits_incident, so the no-evidence path is never more permissive than
the fit-tested path on the same channel.
Recency gates now compare the magnitude of the idle, not the signed value.
The re-correlation sweep anchors "now" to the call's own started_at, so
idle goes negative routinely - incident 9d376ffe recorded
corr_incident_idle_min: -4.1 - and every "idle <= window" test in this
module reads True for a negative number. Those gates had silently stopped
bounding anything for exactly the calls the sweep re-examines.
3. Nothing capped an incident's total size.
Every fit test in the correlator is pairwise: does this call belong with
that incident. Each of f5190670's 68 links was individually arguable; the
mistake was the accumulated shape, which no pairwise rule can see. Two
hard caps now remove an incident from the candidate pool entirely, before
any path can choose it - including the LLM tier, which reads the same
ctx lists.
INCIDENT_MAX_DURATION_MINUTES=120. The one incident in that dump that was
genuinely a single event ran 63 minutes (06:15 wrong-way driver to 07:18
closeout), so the cap has to clear an hour with real headroom. The four
junk chains ran 3h41m, 3h43m, 4h05m and 4h09m, so it has to sit well under
three hours. 120 also equals correlation_window_hours: the location and
slow paths already refuse a candidate older than that, and the fast path
was the only one exempt, so this removes an inconsistency rather than
inventing a number.
INCIDENT_MAX_CALLS=40. A backstop for a burst that fills up inside the
duration cap, not the primary bound. The worst chain averaged ~16
calls/hour while absorbing an entire dispatch backbone, so 40 calls in
under two hours means one incident is eating most of the channel. Set
deliberately above any plausible single-incident call volume (a
multi-alarm fire on its own tactical channel) so this cap errs toward
keeping real incidents whole and lets the duration cap do the cutting.
Capping is not truncation: the incident keeps every call it has and still
auto-resolves on the normal idle sweep. It just stops being a candidate.
Every ambiguous call here was resolved toward a separate incident rather
than a merge. A wrongly-separate incident is visibly wrong and can be
merged later; a wrongly-merged one silently corrupts every unit, tag,
severity and map pin on the incident it joined, and poisons the AI
summary written from them. The cost is some acknowledgements orphaning
instead of riding along on an incident, which is a small, visible loss.
Deliberately NOT changed, since both push toward more merging while the
current failure mode is entirely over-merging (every incident in the dump
has exactly one "new" call; there is no over-splitting left to trade
against):
- unit-overlap positive feedback on shared dispatch channels, which is
now bounded by the caps rather than fixed at its root
- the sweep retry budget expiring before the target incident exists
Tests: 31 new cases in tests/test_correlator_merge_caps.py, including a
replay of the f5190670 night - 13 unrelated jobs at their real offsets,
plus roster unit traffic and acknowledgements every two minutes. Without
the caps that traffic still builds a 125-call incident spanning 244
minutes; with the old thinness test on top, 153 calls over 247 minutes in
3 incidents. With this commit it is 13 incidents, largest 40 calls over 80
minutes. Each new case was checked to fail when the behaviour it covers is
reverted. Suite: 138 passed.
No AI feature flag was touched; correlation stays off in production.
Closes logan/server-26#22
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DRB Server
Full-stack backend for the DRB (Distributed Radio Bot) platform — a community-powered, real-time situational awareness system for public safety radio monitoring. Receives telemetry and audio from SDR edge nodes, runs an AI intelligence pipeline on every call, correlates calls into incidents, dispatches alerts, and serves a live operational dashboard.
What DRB Is
DRB turns radio scanners into a shared geographic intelligence picture. Anyone can contribute coverage by running an edge node (an RTL-SDR dongle on a small Linux machine). All nodes feed into a single platform where operators can:
- Monitor in real time — see active incidents on a map as they unfold, with calls grouped by incident as they come in
- Listen live — stream audio to Discord voice channels or directly in the browser
- Get alerted — receive push notifications when significant events are detected
- Investigate after the fact — search transcribed call history, replay audio, and review incident timelines
The intended deployment is a Tactical Operations Center (TOC) display: a map showing active incidents across all covered jurisdictions, with live audio and alert feeds for operators monitoring an area during an event (severe weather, major incident, etc.).
System Overview
DRB is a distributed SDR (Software Defined Radio) monitoring platform. Edge nodes (small Linux machines with RTL-SDR dongles) decode radio systems and stream audio. The server coordinates those nodes, runs the intelligence pipeline on each call, manages Discord voice bot assignments, and serves the operational dashboard.
Edge Node (client machine)
│
├── MQTT checkin/status/metadata ──► mosquitto ──► c2-core ──► Firestore
│
└── MQTT commands ◄─────────────────────────────── c2-core
│
└── discord_join ──► edge node joins Discord voice + streams PulseAudio
Discord User
└── /join /leave /status /help ──► discord-bot ──► c2-core ──► MQTT ──► edge node
Browser (admin)
└── frontend ──► Firestore (real-time reads, Firebase Auth)
└── c2-core REST API (writes, commands, token pool)
Services
| Service | Container | Description | Port |
|---|---|---|---|
mosquitto |
eclipse-mosquitto |
MQTT broker — receives telemetry from edge nodes, dispatches commands | 1883 |
mosquitto |
eclipse-mosquitto |
MQTT broker — receives telemetry from edge nodes, dispatches commands | 1883 |
c2-core |
Python 3.11 / FastAPI | Command & control API — MQTT handler, intelligence pipeline, incident correlator, alert dispatch, node/system/token management | 8000 |
discord-bot |
Python 3.11 / discord.py | Server-side Discord slash command bot — /join, /leave, /status, /help |
— |
frontend |
Node.js / Next.js 14 | Operational web dashboard — live map, incident feed, call history, node and system management | 3000 |
Directory Structure
Server/
├── docker-compose.yml # Orchestrates all four services
├── Makefile # Convenience targets (setup/build/up/down/logs)
│
├── drb-c2-core/ # FastAPI command & control backend
│ ├── app/
│ │ ├── main.py # App entry point — router registration, MQTT startup, node sweeper
│ │ ├── config.py # Pydantic settings (reads from .env)
│ │ ├── models.py # Shared Pydantic models (CommandPayload, etc.)
│ │ ├── routers/
│ │ │ ├── nodes.py # Node CRUD, approve/reject, command dispatch, system assignment
│ │ │ ├── systems.py # Radio system CRUD (P25/DMR/NBFM configs)
│ │ │ ├── calls.py # Call log retrieval (read-only — calls created by MQTT handler)
│ │ │ ├── tokens.py # Discord bot token pool — add/delete/list; assign_token/release_token helpers
│ │ │ ├── upload.py # Audio file upload endpoint (called by edge nodes); triggers intelligence pipeline
│ │ │ ├── incidents.py # Incident CRUD, manual link/unlink calls, resolve
│ │ │ └── alerts.py # Alert rule CRUD
│ │ └── internal/
│ │ ├── auth.py # Auth — Firebase ID token OR service key (Bearer)
│ │ ├── firestore.py # Async Firestore wrappers (doc_get, doc_set, doc_update, collection_list)
│ │ ├── mqtt_handler.py # MQTT subscriber — call_start/call_end handlers, node checkin/status
│ │ ├── node_sweeper.py # Background task — marks nodes offline after 90s without heartbeat
│ │ ├── storage.py # GCS audio file uploads
│ │ ├── transcription.py # STT pipeline — OpenAI Whisper / GPT-4o transcribe
│ │ ├── intelligence.py # GPT scene extraction: tags, incident type, location, units, vehicles; geocoding
│ │ ├── incident_correlator.py # Hybrid correlator — matches calls to incidents or creates new ones
│ │ └── alerter.py # Alert dispatch — evaluates rules, sends notifications
│ ├── scripts/
│ │ └── set_admin.py # CLI to grant/revoke Firebase admin custom claim
│ ├── gcp-key.json # GCP service account key — place here, NOT committed to git
│ └── .env.example
│
├── drb-server-discord-bot/ # Discord slash command bot (server-side)
│ ├── app/
│ │ ├── main.py # Bot startup, cog loading, guild sync
│ │ ├── config.py # Settings (Discord token, C2 URL, service key)
│ │ ├── commands/
│ │ │ └── radio.py # /join (with token autocomplete), /leave, /status, /help
│ │ └── internal/
│ │ └── c2_client.py # Async HTTP client to c2-core — includes service key auth header
│ └── .env.example
│
└── drb-frontend/ # Next.js 14 admin dashboard
├── app/
│ ├── dashboard/ # Overview: active node grid + live call feed
│ ├── map/ # Leaflet map — node locations, active incident pins
│ ├── calls/ # Call history — talkgroup, duration, audio playback, transcript
│ ├── nodes/ # Node list + per-node detail (approve, reject, assign system)
│ ├── systems/ # Radio system CRUD (P25 form with talkgroup editor, DMR/NBFM JSON)
│ ├── tokens/ # Discord bot token pool management
│ ├── incidents/ # Incident list/detail — linked calls, location, summary, resolve
│ ├── alerts/ # Alert rule configuration
│ └── login/ # Firebase Auth login page
├── components/
│ ├── MapView.tsx # react-leaflet map with status-colored markers
│ ├── NodeCard.tsx # Node status card with action buttons
│ ├── NodeConfigModal.tsx # Modal for assigning a radio system to a node
│ ├── CallRow.tsx # Call history row — talkgroup, duration, audio player
│ ├── AuthProvider.tsx # Firebase Auth context + middleware guard
│ ├── Nav.tsx # Sidebar navigation
│ └── StatusBadge.tsx # Color-coded status pill (online/recording/offline/unconfigured)
├── lib/
│ ├── firebase.ts # Firebase JS SDK initialization
│ ├── c2api.ts # Fetch wrapper for c2-core REST API (attaches Firebase ID token)
│ ├── types.ts # Shared TypeScript types (NodeRecord, CallRecord, SystemRecord, etc.)
│ ├── useNodes.ts # Firestore real-time subscription to nodes collection
│ ├── useCalls.ts # Firestore real-time subscription to calls (active + history)
│ └── useSystems.ts # Firestore real-time subscription to systems collection
├── next.config.mjs # Next.js config — standalone output, transpilePackages for Leaflet
└── .env.example
Prerequisites
- Docker + Docker Compose
- A Firebase project with:
- Firestore enabled (Native mode)
- Firebase Authentication enabled (Email/Password provider)
- A GCP service account key with Firestore + Firebase Auth Admin SDK permissions
- A Discord application with:
- One bot token for the server command bot (handles
/join,/leave, etc.) - One or more additional bot tokens for the pool (edge nodes use these to join voice channels)
- One bot token for the server command bot (handles
Setup
# 1. Copy all env files
make setup
# 2. Place your GCP service account key
cp /path/to/your-service-account-key.json drb-c2-core/gcp-key.json
# 3. Fill in secrets — see tables below
nano drb-c2-core/.env
nano drb-server-discord-bot/.env
nano drb-frontend/.env
# 4. Build and start
make build
make up
# 5. Grant admin access to your Firebase user
cd drb-c2-core
python scripts/set_admin.py grant your@email.com
# Sign out and back in to the frontend for the claim to take effect
Environment Variables
drb-c2-core/.env
| Variable | Required | Description |
|---|---|---|
MQTT_BROKER |
Yes | Hostname of the MQTT broker (default: mosquitto) |
MQTT_PORT |
No | MQTT broker port (default: 1883) |
FIRESTORE_DATABASE |
No | Firestore database name (default: (default)) |
GCP_CREDENTIALS_PATH |
Yes | Path to the service account key inside the container (e.g. /secrets/gcp-key.json) |
GCS_BUCKET |
No | GCS bucket name for storing call audio recordings |
SERVICE_KEY |
Yes* | Shared secret for internal service-to-service auth. Must match C2_SERVICE_KEY in the discord-bot env |
*Required for the server Discord bot to authenticate against the C2 API.
drb-server-discord-bot/.env
| Variable | Required | Description |
|---|---|---|
DISCORD_TOKEN |
Yes | Bot token for the server-side slash command bot |
C2_URL |
Yes | Internal URL of c2-core (e.g. http://c2-core:8000) |
C2_SERVICE_KEY |
Yes* | Must match SERVICE_KEY in drb-c2-core — sent as a Bearer token on all C2 API calls |
DEV_GUILD_ID |
No | Discord guild ID to sync slash commands instantly during development |
drb-frontend/.env
| Variable | Required | Description |
|---|---|---|
NEXT_PUBLIC_FIREBASE_API_KEY |
Yes | Firebase web API key |
NEXT_PUBLIC_FIREBASE_AUTH_DOMAIN |
Yes | Firebase auth domain |
NEXT_PUBLIC_FIREBASE_PROJECT_ID |
Yes | GCP/Firebase project ID |
NEXT_PUBLIC_FIREBASE_STORAGE_BUCKET |
Yes | Firebase storage bucket |
NEXT_PUBLIC_FIREBASE_MESSAGING_SENDER_ID |
Yes | Firebase messaging sender ID |
NEXT_PUBLIC_FIREBASE_APP_ID |
Yes | Firebase app ID |
NEXT_PUBLIC_FIRESTORE_DATABASE |
No | Firestore database name — must match c2-core (default: (default)) |
NEXT_PUBLIC_C2_URL |
Yes | C2 API URL reachable from the user's browser |
Authentication
Two auth mechanisms are supported on all C2 routes:
- Firebase ID tokens — issued by Firebase Auth to browser users. The frontend attaches these automatically via
c2api.ts. Required for the dashboard to call the C2 API. - Service key — a static shared secret for internal service calls. The server Discord bot sends this as a
Bearertoken. SetSERVICE_KEYin c2-core andC2_SERVICE_KEYin the discord-bot to the same value.
Admin-only routes (node approve/reject) additionally require the Firebase user to have an admin: true custom claim:
python drb-c2-core/scripts/set_admin.py grant your@email.com
python drb-c2-core/scripts/set_admin.py revoke your@email.com
Discord Bot Token Pool
Edge nodes join Discord voice channels using bot tokens managed by the server. Add tokens in the Tokens page of the frontend (or POST /tokens). Each token has:
- A friendly name
- The actual Discord bot token string (masked in list views)
in_useflag and the node it's currently assigned to
Flow when a user runs /join:
- Discord bot sends a
discord_joincommand to c2-core with the target guild and channel IDs. - c2-core picks a free token (optionally the one requested via autocomplete), marks it
in_use, and forwards the token + channel info to the target edge node over MQTT. - The edge node starts a
discord.pybot with that token and joins the voice channel. - The bot streams live audio continuously from PulseAudio (
drb_sink.monitoron the edge node). Icecast is for browser listening only. - On
/leave, c2-core releases the token back to the pool.
Node Lifecycle
- Edge node starts → sends MQTT
checkin→ appears as pending in the dashboard - Admin clicks Approve → c2-core generates a random API key, stores it in
node_keys, publishes it to the node over MQTT, updates node toapproved - Admin assigns a radio system via Config → c2-core pushes the full system config (frequencies, talkgroups, Icecast settings) over MQTT
- Edge node writes the config and (re)starts OP25
- Node sends
online/recording/unconfiguredstatus via 30s MQTT heartbeats node_sweeperbackground task marks any node offline after 90s without a heartbeat
Call & Intelligence Pipeline
Every radio call flows through a four-stage pipeline triggered when the edge node uploads its recording:
Edge node ──► MQTT call_start ──► CallRecord created (active)
│
Edge node ──► audio upload ──► GCS storage
│
▼
[1] TRANSCRIPTION
OpenAI Whisper / GPT-4o transcribe
→ CallRecord.transcript
│
▼
[2] INTELLIGENCE EXTRACTION (GPT-4o-mini)
Scene detection — splits multi-incident recordings
Speaker role inference — dispatch vs. unit patterns
used to correctly attribute locations (dispatch-
provided address vs. unit position report) and
units (being dispatched vs. acknowledging)
Entity extraction: tags, incident_type, location,
units, vehicles, severity, resolved flag
+ geocoding (Google Maps)
+ embedding (text-embedding-3-small)
→ CallRecord.tags, .location, .units, etc.
│
▼
[3] INCIDENT CORRELATION (hybrid engine)
Fast path — talkgroup + recency
Unit path — same officer continuity
Location — proximity match
Cross-TG — multi-agency / channel hop
Slow path — embedding similarity
→ IncidentRecord created or updated
│
▼
[4] ALERT DISPATCH
Evaluate alert rules (keywords, talkgroups)
→ notifications sent
Frontend Pages
| Page | URL | Description |
|---|---|---|
| Dashboard | /dashboard |
Live node grid + active call stream |
| Map | /map |
Leaflet map — nodes color-coded by status, active call popups |
| Calls | /calls |
Full call history — talkgroup, duration, audio playback, transcript |
| Nodes | /nodes |
Node list; per-node detail for approve/reject/assign system |
| Systems | /systems |
Create and manage P25/DMR/NBFM radio system configurations |
| Tokens | /tokens |
Discord bot token pool management |
| Incidents | /incidents |
Incident list/detail — linked calls, location, summary, tags, resolve |
| Alerts | /alerts |
Not built |
Makefile Targets
make setup — copy .env.example files to .env
make build — docker compose build
make up — docker compose up -d
make down — docker compose down
make logs — follow all service logs
make logs-c2 — c2-core logs only
make logs-bot — discord-bot logs only
make logs-frontend — frontend logs only