app/internal/incident_correlator.py
- *`correlate_call`* — added units and vehicles optional params; when provided (per-scene from intelligence extraction), they take priority over the merged call-document values, preventing multi-scene unit contamination
- *Cross-TGID correlation path (2.5)* — *new path between location and slow paths*: when a call shares 2+ unit IDs with a recent same-system, same-type incident AND embedding similarity ≥ 0.85, it links them — catches multi-talkgroup pursuits like the bicycle search that split across dispatch/tactical/geographic channels
# `app/internal/intelligence.py`
- *`reassignment` field* — added to the GPT-4o-mini prompt schema and rules; `true` when dispatch is actively pulling a unit to a new, different call (not a status update or en route acknowledgement); returned in every processed scene dict
- *Tag location rule* — added explicit instruction to the prompt: tags must describe what happened, not where; place names, road names, and talkgroup names are explicitly forbidden as tags
# `app/routers/upload.py`
- Both scene correlation call sites (`_run_extraction_pipeline` and `_run_intelligence_pipeline`) now pass `units=corr_units` where `corr_units = [] if scene.get("reassignment") else scene.get("units") `— suppresses unit overlap matching when a unit is being reassigned to a new call, preventing chaining into their previous incident
- Both sites also pass `vehicles=scene.get("vehicles")` (per-scene vehicles, from the multi-scene units fix)
# `app/config.py`
- `embedding_cross_tg_threshold: float = 0.85` — threshold for the new cross-TGID path
DRB Server
Full-stack backend for the Discord Radio Bot (DRB) system. Receives telemetry from SDR edge nodes over MQTT, stores data in Firestore, orchestrates Discord voice bots through a token pool, and serves a real-time admin dashboard.
System Overview
DRB is a distributed SDR (Software Defined Radio) monitoring platform. Edge nodes (small Linux machines with RTL-SDR dongles) decode radio systems and stream audio. The server coordinates those nodes, manages which Discord voice channels receive audio, and stores call history.
Edge Node (client machine)
│
├── MQTT checkin/status/metadata ──► mosquitto ──► c2-core ──► Firestore
│
└── MQTT commands ◄─────────────────────────────── c2-core
│
└── discord_join ──► edge node joins Discord voice + streams PulseAudio
Discord User
└── /join /leave /status /help ──► discord-bot ──► c2-core ──► MQTT ──► edge node
Browser (admin)
└── frontend ──► Firestore (real-time reads, Firebase Auth)
└── c2-core REST API (writes, commands, token pool)
Services
| Service | Container | Description | Port |
|---|---|---|---|
mosquitto |
eclipse-mosquitto |
MQTT broker — receives telemetry from edge nodes, dispatches commands | 1883 |
c2-core |
Python 3.11 / FastAPI | Command & control API — MQTT subscriber, Firestore writes, node/system/call/token management | 8000 |
discord-bot |
Python 3.11 / discord.py | Server-side Discord slash command bot — /join, /leave, /status, /help |
— |
frontend |
Node.js / Next.js 14 | Admin web dashboard — real-time node map, call logs, system & token management | 3000 |
Directory Structure
Server/
├── docker-compose.yml # Orchestrates all four services
├── Makefile # Convenience targets (setup/build/up/down/logs)
│
├── drb-c2-core/ # FastAPI command & control backend
│ ├── app/
│ │ ├── main.py # App entry point — router registration, MQTT startup, node sweeper
│ │ ├── config.py # Pydantic settings (reads from .env)
│ │ ├── models.py # Shared Pydantic models (CommandPayload, etc.)
│ │ ├── routers/
│ │ │ ├── nodes.py # Node CRUD, approve/reject, command dispatch, system assignment
│ │ │ ├── systems.py # Radio system CRUD (P25/DMR/NBFM configs)
│ │ │ ├── calls.py # Call log retrieval (read-only — calls created by MQTT handler)
│ │ │ ├── tokens.py # Discord bot token pool — add/delete/list; assign_token/release_token helpers
│ │ │ ├── upload.py # Audio file upload endpoint (called by edge nodes)
│ │ │ ├── incidents.py # [PLANNED] Incident CRUD, manual link/unlink calls, resolve
│ │ │ └── alerts.py # [PLANNED] Alert rule CRUD
│ │ └── internal/
│ │ ├── auth.py # Auth — Firebase ID token OR service key (Bearer)
│ │ ├── firestore.py # Async Firestore wrappers (doc_get, doc_set, doc_update, collection_list)
│ │ ├── mqtt_handler.py # MQTT subscriber — call_start/call_end handlers, node checkin/status
│ │ ├── node_sweeper.py # Background task — marks nodes offline after 90s without heartbeat
│ │ ├── storage.py # GCS audio file uploads
│ │ ├── transcription.py # [PLANNED] STT pipeline — Whisper or Google Speech API
│ │ ├── intelligence.py # [PLANNED] Keyword/entity extraction, severity scoring
│ │ ├── incident_correlator.py # [PLANNED] Match calls to incidents or create new ones
│ │ └── alerter.py # [PLANNED] Alert dispatch (Discord webhook, push, etc.)
│ ├── scripts/
│ │ └── set_admin.py # CLI to grant/revoke Firebase admin custom claim
│ ├── gcp-key.json # GCP service account key — place here, NOT committed to git
│ └── .env.example
│
├── drb-server-discord-bot/ # Discord slash command bot (server-side)
│ ├── app/
│ │ ├── main.py # Bot startup, cog loading, guild sync
│ │ ├── config.py # Settings (Discord token, C2 URL, service key)
│ │ ├── commands/
│ │ │ └── radio.py # /join (with token autocomplete), /leave, /status, /help
│ │ └── internal/
│ │ └── c2_client.py # Async HTTP client to c2-core — includes service key auth header
│ └── .env.example
│
└── drb-frontend/ # Next.js 14 admin dashboard
├── app/
│ ├── dashboard/ # Overview: active node grid + live call feed
│ ├── map/ # Leaflet map — node locations + [PLANNED] incident pins
│ ├── calls/ # Call history — talkgroup, duration, audio, transcript (when populated)
│ ├── nodes/ # Node list + per-node detail (approve, reject, assign system)
│ ├── systems/ # Radio system CRUD (P25 form with talkgroup editor, DMR/NBFM JSON)
│ ├── tokens/ # Discord bot token pool management
│ ├── incidents/ # [PLANNED] Incident list/detail — linked calls, location, resolve
│ ├── alerts/ # [PLANNED] Alert rule configuration
│ └── login/ # Firebase Auth login page
├── components/
│ ├── MapView.tsx # react-leaflet map with status-colored markers
│ ├── NodeCard.tsx # Node status card with action buttons
│ ├── NodeConfigModal.tsx # Modal for assigning a radio system to a node
│ ├── CallRow.tsx # Call history row — talkgroup, duration, audio player
│ ├── AuthProvider.tsx # Firebase Auth context + middleware guard
│ ├── Nav.tsx # Sidebar navigation
│ └── StatusBadge.tsx # Color-coded status pill (online/recording/offline/unconfigured)
├── lib/
│ ├── firebase.ts # Firebase JS SDK initialization
│ ├── c2api.ts # Fetch wrapper for c2-core REST API (attaches Firebase ID token)
│ ├── types.ts # Shared TypeScript types (NodeRecord, CallRecord, SystemRecord, etc.)
│ ├── useNodes.ts # Firestore real-time subscription to nodes collection
│ ├── useCalls.ts # Firestore real-time subscription to calls (active + history)
│ └── useSystems.ts # Firestore real-time subscription to systems collection
├── next.config.mjs # Next.js config — standalone output, transpilePackages for Leaflet
└── .env.example
Prerequisites
- Docker + Docker Compose
- A Firebase project with:
- Firestore enabled (Native mode)
- Firebase Authentication enabled (Email/Password provider)
- A GCP service account key with Firestore + Firebase Auth Admin SDK permissions
- A Discord application with:
- One bot token for the server command bot (handles
/join,/leave, etc.) - One or more additional bot tokens for the pool (edge nodes use these to join voice channels)
- One bot token for the server command bot (handles
Setup
# 1. Copy all env files
make setup
# 2. Place your GCP service account key
cp /path/to/your-service-account-key.json drb-c2-core/gcp-key.json
# 3. Fill in secrets — see tables below
nano drb-c2-core/.env
nano drb-server-discord-bot/.env
nano drb-frontend/.env
# 4. Build and start
make build
make up
# 5. Grant admin access to your Firebase user
cd drb-c2-core
python scripts/set_admin.py grant your@email.com
# Sign out and back in to the frontend for the claim to take effect
Environment Variables
drb-c2-core/.env
| Variable | Required | Description |
|---|---|---|
MQTT_BROKER |
Yes | Hostname of the MQTT broker (default: mosquitto) |
MQTT_PORT |
No | MQTT broker port (default: 1883) |
FIRESTORE_DATABASE |
No | Firestore database name (default: (default)) |
GCP_CREDENTIALS_PATH |
Yes | Path to the service account key inside the container (e.g. /secrets/gcp-key.json) |
GCS_BUCKET |
No | GCS bucket name for storing call audio recordings |
SERVICE_KEY |
Yes* | Shared secret for internal service-to-service auth. Must match C2_SERVICE_KEY in the discord-bot env |
*Required for the server Discord bot to authenticate against the C2 API.
drb-server-discord-bot/.env
| Variable | Required | Description |
|---|---|---|
DISCORD_TOKEN |
Yes | Bot token for the server-side slash command bot |
C2_URL |
Yes | Internal URL of c2-core (e.g. http://c2-core:8000) |
C2_SERVICE_KEY |
Yes* | Must match SERVICE_KEY in drb-c2-core — sent as a Bearer token on all C2 API calls |
DEV_GUILD_ID |
No | Discord guild ID to sync slash commands instantly during development |
drb-frontend/.env
| Variable | Required | Description |
|---|---|---|
NEXT_PUBLIC_FIREBASE_API_KEY |
Yes | Firebase web API key |
NEXT_PUBLIC_FIREBASE_AUTH_DOMAIN |
Yes | Firebase auth domain |
NEXT_PUBLIC_FIREBASE_PROJECT_ID |
Yes | GCP/Firebase project ID |
NEXT_PUBLIC_FIREBASE_STORAGE_BUCKET |
Yes | Firebase storage bucket |
NEXT_PUBLIC_FIREBASE_MESSAGING_SENDER_ID |
Yes | Firebase messaging sender ID |
NEXT_PUBLIC_FIREBASE_APP_ID |
Yes | Firebase app ID |
NEXT_PUBLIC_FIRESTORE_DATABASE |
No | Firestore database name — must match c2-core (default: (default)) |
NEXT_PUBLIC_C2_URL |
Yes | C2 API URL reachable from the user's browser |
Authentication
Two auth mechanisms are supported on all C2 routes:
- Firebase ID tokens — issued by Firebase Auth to browser users. The frontend attaches these automatically via
c2api.ts. Required for the dashboard to call the C2 API. - Service key — a static shared secret for internal service calls. The server Discord bot sends this as a
Bearertoken. SetSERVICE_KEYin c2-core andC2_SERVICE_KEYin the discord-bot to the same value.
Admin-only routes (node approve/reject) additionally require the Firebase user to have an admin: true custom claim:
python drb-c2-core/scripts/set_admin.py grant your@email.com
python drb-c2-core/scripts/set_admin.py revoke your@email.com
Discord Bot Token Pool
Edge nodes join Discord voice channels using bot tokens managed by the server. Add tokens in the Tokens page of the frontend (or POST /tokens). Each token has:
- A friendly name
- The actual Discord bot token string (masked in list views)
in_useflag and the node it's currently assigned to
Flow when a user runs /join:
- Discord bot sends a
discord_joincommand to c2-core with the target guild and channel IDs. - c2-core picks a free token (optionally the one requested via autocomplete), marks it
in_use, and forwards the token + channel info to the target edge node over MQTT. - The edge node starts a
discord.pybot with that token and joins the voice channel. - The bot streams live audio continuously from PulseAudio (
drb_sink.monitoron the edge node). Icecast is for browser listening only. - On
/leave, c2-core releases the token back to the pool.
Node Lifecycle
- Edge node starts → sends MQTT
checkin→ appears as pending in the dashboard - Admin clicks Approve → c2-core generates a random API key, stores it in
node_keys, publishes it to the node over MQTT, updates node toapproved - Admin assigns a radio system via Config → c2-core pushes the full system config (frequencies, talkgroups, Icecast settings) over MQTT
- Edge node writes the config and (re)starts OP25
- Node sends
online/recording/unconfiguredstatus via 30s MQTT heartbeats node_sweeperbackground task marks any node offline after 90s without a heartbeat
Call & Intelligence Pipeline
This is the full intended data lifecycle for every radio call — from raw RF to searchable, cross-referenced intelligence. The data models and Firestore schema are already designed for this; several pipeline stages are stubs awaiting implementation.
Call lifecycle (current — fully working)
Edge node ──► MQTT call_start ──► c2-core creates CallRecord (status: active)
│ talkgroup_id, talkgroup_name, freq,
│ node_id, system_id, started_at
▼
Firestore "calls" collection
│
▼
Frontend live call feed / map popups
│
Edge node ──► MQTT call_end ──► c2-core updates CallRecord (status: ended)
│ ended_at, audio_url (GCS link)
▼
Frontend call history / audio playback
Intelligence pipeline (designed — implementation pending)
After a call ends, the following stages should fire in order:
CallRecord (ended, audio_url set)
│
▼
[1] TRANSCRIPTION
Speech-to-text on the GCS audio file (Whisper or Google Speech-to-Text)
→ writes CallRecord.transcript
│
▼
[2] INTELLIGENCE EXTRACTION
Analyze transcript for:
- Named entities: unit IDs, street addresses, location references
- Keywords / keyword sets: fire/EMS/police/hazmat/pursuit/shots fired/etc.
- Radio codes (10-codes, signals) mapped to plain English
- Severity scoring
→ writes CallRecord.tags[], CallRecord.location (geocoded if address found)
│
▼
[3] INCIDENT CORRELATION
Given the extracted entities + tags, decide:
- Does this call match an existing active IncidentRecord?
(same location ± radius, overlapping tags, recent time window)
→ link: append CallRecord.call_id to IncidentRecord.call_ids,
set CallRecord.incident_id, update IncidentRecord.summary
- Or does this call describe a new event?
→ create IncidentRecord (type, location, title, tags, status: active)
link the call, set CallRecord.incident_id
│
▼
[4] ALERTS
If incident is new OR severity exceeds threshold:
- Trigger configured alert channels (Discord webhook, push notification, etc.)
- Include: incident type, location, talkgroup, transcript excerpt
Data model fields involved
CallRecord (Firestore collection: calls):
| Field | Type | Populated by |
|---|---|---|
call_id |
string | MQTT handler on call_start |
node_id |
string | MQTT handler |
system_id |
string | MQTT handler (from node's assigned system) |
talkgroup_id |
number | MQTT handler |
talkgroup_name |
string | MQTT handler |
freq |
number | MQTT handler |
started_at |
timestamp | MQTT handler |
ended_at |
timestamp | MQTT handler on call_end |
status |
active | ended |
MQTT handler |
audio_url |
string | Edge node upload → GCS |
transcript |
string | null | [stub] Transcription pipeline |
incident_id |
string | null | [stub] Incident correlation |
location |
{lat, lng} | null |
[stub] Geocoding from transcript |
tags |
string[] | [stub] Intelligence extraction |
IncidentRecord (Firestore collection: incidents):
| Field | Type | Description |
|---|---|---|
incident_id |
string | UUID |
title |
string | Auto-generated or manually set |
type |
string | fire, ems, police, hazmat, pursuit, etc. |
status |
active | resolved |
Updated as calls accumulate or manually resolved |
location |
{lat, lng} |
Geocoded from first call with a location |
call_ids |
string[] | All linked CallRecord IDs |
started_at |
timestamp | Timestamp of first linked call |
updated_at |
timestamp | Updated on each new linked call |
summary |
string | null | Auto-generated from transcripts or manually written |
tags |
string[] | Union of tags from all linked calls |
Backend — what needs to be built
The following do not exist yet and need to be created:
drb-c2-core/app/routers/incidents.py— CRUD + manual incident managementdrb-c2-core/app/routers/alerts.py— Alert rule configurationdrb-c2-core/app/internal/transcription.py— STT integration (Whisper local or Google Speech API)drb-c2-core/app/internal/intelligence.py— Keyword/entity extraction, severity scoringdrb-c2-core/app/internal/incident_correlator.py— Match calls to incidents or create new onesdrb-c2-core/app/internal/alerter.py— Dispatch alerts (Discord webhook, etc.)
The _on_call_end() handler in mqtt_handler.py is the natural trigger point — after updating the CallRecord, it should enqueue the transcription + intelligence pipeline.
Frontend — what needs to be built
The following pages and nav items are not yet implemented:
/incidents— Incident list and detail view (linked calls, map pin, transcript summary, tags, resolve button)/alerts— Alert rule configuration (keyword sets, talkgroup filters, notification channels)Nav.tsxshould add Calls and Incidents as primary nav items; the current Calls page exists but isn't in the design's intended nav hierarchy
The Map page should eventually show both node markers and incident pins — incidents with location set should appear as color-coded markers (by type/severity) alongside the node status markers.
Frontend Pages
| Page | URL | Status | Description |
|---|---|---|---|
| Dashboard | /dashboard |
Working | Live node grid + active call stream |
| Map | /map |
Working | Leaflet map — nodes color-coded by status, active call popups |
| Calls | /calls |
Working | Full call history — talkgroup, duration, audio playback, transcript (when populated) |
| Nodes | /nodes |
Working | Node list; per-node detail for approve/reject/assign system |
| Systems | /systems |
Working | Create and manage P25/DMR/NBFM radio system configurations |
| Tokens | /tokens |
Working | Discord bot token pool management |
| Incidents | /incidents |
Not built | Incident list/detail — linked calls, location, summary, tags, resolve |
| Alerts | /alerts |
Not built | Alert rule configuration — keywords, talkgroups, notification channels |
Makefile Targets
make setup — copy .env.example files to .env
make build — docker compose build
make up — docker compose up -d
make down — docker compose down
make logs — follow all service logs
make logs-c2 — c2-core logs only
make logs-bot — discord-bot logs only
make logs-frontend — frontend logs only