server-26/README.md

# DRB Server

Full-stack backend for the Discord Radio Bot (DRB) system. Receives telemetry from SDR edge nodes over MQTT, stores data in Firestore, orchestrates Discord voice bots through a token pool, and serves a real-time admin dashboard.

## System Overview

DRB is a distributed SDR (Software Defined Radio) monitoring platform. Edge nodes (small Linux machines with RTL-SDR dongles) decode radio systems and stream audio. The server coordinates those nodes, manages which Discord voice channels receive audio, and stores call history.

```
Edge Node (client machine)
    │
    ├── MQTT checkin/status/metadata ──► mosquitto ──► c2-core ──► Firestore
    │
    └── MQTT commands ◄─────────────────────────────── c2-core
              │
              └── discord_join ──► edge node joins Discord voice + streams Icecast

Discord User
    └── /join /leave /status /help ──► discord-bot ──► c2-core ──► MQTT ──► edge node

Browser (admin)
    └── frontend ──► Firestore (real-time reads, Firebase Auth)
                └── c2-core REST API (writes, commands, token pool)
```

## Services

| Service | Container | Description | Port |
|---|---|---|---|
| `mosquitto` | `eclipse-mosquitto` | MQTT broker — receives telemetry from edge nodes, dispatches commands | 1883 |
| `c2-core` | Python 3.11 / FastAPI | Command & control API — MQTT subscriber, Firestore writes, node/system/call/token management | 8000 |
| `discord-bot` | Python 3.11 / discord.py | Server-side Discord slash command bot — `/join`, `/leave`, `/status`, `/help` | — |
| `frontend` | Node.js / Next.js 14 | Admin web dashboard — real-time node map, call logs, system & token management | 3000 |

## Directory Structure

```
Server/
├── docker-compose.yml              # Orchestrates all four services
├── Makefile                        # Convenience targets (setup/build/up/down/logs)
│
├── drb-c2-core/                    # FastAPI command & control backend
│   ├── app/
│   │   ├── main.py                 # App entry point — router registration, MQTT startup, node sweeper
│   │   ├── config.py               # Pydantic settings (reads from .env)
│   │   ├── models.py               # Shared Pydantic models (CommandPayload, etc.)
│   │   ├── routers/
│   │   │   ├── nodes.py            # Node CRUD, approve/reject, command dispatch, system assignment
│   │   │   ├── systems.py          # Radio system CRUD (P25/DMR/NBFM configs)
│   │   │   ├── calls.py            # Call log retrieval (read-only — calls created by MQTT handler)
│   │   │   ├── tokens.py           # Discord bot token pool — add/delete/list; assign_token/release_token helpers
│   │   │   ├── upload.py           # Audio file upload endpoint (called by edge nodes)
│   │   │   ├── incidents.py        # [PLANNED] Incident CRUD, manual link/unlink calls, resolve
│   │   │   └── alerts.py           # [PLANNED] Alert rule CRUD
│   │   └── internal/
│   │       ├── auth.py             # Auth — Firebase ID token OR service key (Bearer)
│   │       ├── firestore.py        # Async Firestore wrappers (doc_get, doc_set, doc_update, collection_list)
│   │       ├── mqtt_handler.py     # MQTT subscriber — call_start/call_end handlers, node checkin/status
│   │       ├── node_sweeper.py     # Background task — marks nodes offline after 90s without heartbeat
│   │       ├── storage.py          # GCS audio file uploads
│   │       ├── transcription.py    # [PLANNED] STT pipeline — Whisper or Google Speech API
│   │       ├── intelligence.py     # [PLANNED] Keyword/entity extraction, severity scoring
│   │       ├── incident_correlator.py # [PLANNED] Match calls to incidents or create new ones
│   │       └── alerter.py          # [PLANNED] Alert dispatch (Discord webhook, push, etc.)
│   ├── scripts/
│   │   └── set_admin.py            # CLI to grant/revoke Firebase admin custom claim
│   ├── gcp-key.json                # GCP service account key — place here, NOT committed to git
│   └── .env.example
│
├── drb-server-discord-bot/         # Discord slash command bot (server-side)
│   ├── app/
│   │   ├── main.py                 # Bot startup, cog loading, guild sync
│   │   ├── config.py               # Settings (Discord token, C2 URL, service key)
│   │   ├── commands/
│   │   │   └── radio.py            # /join (with token autocomplete), /leave, /status, /help
│   │   └── internal/
│   │       └── c2_client.py        # Async HTTP client to c2-core — includes service key auth header
│   └── .env.example
│
└── drb-frontend/                   # Next.js 14 admin dashboard
    ├── app/
    │   ├── dashboard/              # Overview: active node grid + live call feed
    │   ├── map/                    # Leaflet map — node locations + [PLANNED] incident pins
    │   ├── calls/                  # Call history — talkgroup, duration, audio, transcript (when populated)
    │   ├── nodes/                  # Node list + per-node detail (approve, reject, assign system)
    │   ├── systems/                # Radio system CRUD (P25 form with talkgroup editor, DMR/NBFM JSON)
    │   ├── tokens/                 # Discord bot token pool management
    │   ├── incidents/              # [PLANNED] Incident list/detail — linked calls, location, resolve
    │   ├── alerts/                 # [PLANNED] Alert rule configuration
    │   └── login/                  # Firebase Auth login page
    ├── components/
    │   ├── MapView.tsx             # react-leaflet map with status-colored markers
    │   ├── NodeCard.tsx            # Node status card with action buttons
    │   ├── NodeConfigModal.tsx     # Modal for assigning a radio system to a node
    │   ├── CallRow.tsx             # Call history row — talkgroup, duration, audio player
    │   ├── AuthProvider.tsx        # Firebase Auth context + middleware guard
    │   ├── Nav.tsx                 # Sidebar navigation
    │   └── StatusBadge.tsx         # Color-coded status pill (online/recording/offline/unconfigured)
    ├── lib/
    │   ├── firebase.ts             # Firebase JS SDK initialization
    │   ├── c2api.ts                # Fetch wrapper for c2-core REST API (attaches Firebase ID token)
    │   ├── types.ts                # Shared TypeScript types (NodeRecord, CallRecord, SystemRecord, etc.)
    │   ├── useNodes.ts             # Firestore real-time subscription to nodes collection
    │   ├── useCalls.ts             # Firestore real-time subscription to calls (active + history)
    │   └── useSystems.ts           # Firestore real-time subscription to systems collection
    ├── next.config.mjs             # Next.js config — standalone output, transpilePackages for Leaflet
    └── .env.example
```

## Prerequisites

- Docker + Docker Compose
- A **Firebase project** with:
  - Firestore enabled (Native mode)
  - Firebase Authentication enabled (Email/Password provider)
  - A GCP service account key with Firestore + Firebase Auth Admin SDK permissions
- A **Discord application** with:
  - One bot token for the server command bot (handles `/join`, `/leave`, etc.)
  - One or more additional bot tokens for the pool (edge nodes use these to join voice channels)

## Setup

```bash
# 1. Copy all env files
make setup

# 2. Place your GCP service account key
cp /path/to/your-service-account-key.json drb-c2-core/gcp-key.json

# 3. Fill in secrets — see tables below
nano drb-c2-core/.env
nano drb-server-discord-bot/.env
nano drb-frontend/.env

# 4. Build and start
make build
make up

# 5. Grant admin access to your Firebase user
cd drb-c2-core
python scripts/set_admin.py grant your@email.com
# Sign out and back in to the frontend for the claim to take effect
```

## Environment Variables

### `drb-c2-core/.env`

| Variable | Required | Description |
|---|---|---|
| `MQTT_BROKER` | Yes | Hostname of the MQTT broker (default: `mosquitto`) |
| `MQTT_PORT` | No | MQTT broker port (default: `1883`) |
| `FIRESTORE_DATABASE` | No | Firestore database name (default: `(default)`) |
| `GCP_CREDENTIALS_PATH` | Yes | Path to the service account key inside the container (e.g. `/secrets/gcp-key.json`) |
| `GCS_BUCKET` | No | GCS bucket name for storing call audio recordings |
| `SERVICE_KEY` | Yes* | Shared secret for internal service-to-service auth. Must match `C2_SERVICE_KEY` in the discord-bot env |

*Required for the server Discord bot to authenticate against the C2 API.

### `drb-server-discord-bot/.env`

| Variable | Required | Description |
|---|---|---|
| `DISCORD_TOKEN` | Yes | Bot token for the server-side slash command bot |
| `C2_URL` | Yes | Internal URL of c2-core (e.g. `http://c2-core:8000`) |
| `C2_SERVICE_KEY` | Yes* | Must match `SERVICE_KEY` in drb-c2-core — sent as a Bearer token on all C2 API calls |
| `DEV_GUILD_ID` | No | Discord guild ID to sync slash commands instantly during development |

### `drb-frontend/.env`

| Variable | Required | Description |
|---|---|---|
| `NEXT_PUBLIC_FIREBASE_API_KEY` | Yes | Firebase web API key |
| `NEXT_PUBLIC_FIREBASE_AUTH_DOMAIN` | Yes | Firebase auth domain |
| `NEXT_PUBLIC_FIREBASE_PROJECT_ID` | Yes | GCP/Firebase project ID |
| `NEXT_PUBLIC_FIREBASE_STORAGE_BUCKET` | Yes | Firebase storage bucket |
| `NEXT_PUBLIC_FIREBASE_MESSAGING_SENDER_ID` | Yes | Firebase messaging sender ID |
| `NEXT_PUBLIC_FIREBASE_APP_ID` | Yes | Firebase app ID |
| `NEXT_PUBLIC_FIRESTORE_DATABASE` | No | Firestore database name — must match c2-core (default: `(default)`) |
| `NEXT_PUBLIC_C2_URL` | Yes | C2 API URL reachable from the user's browser |

## Authentication

Two auth mechanisms are supported on all C2 routes:

1. **Firebase ID tokens** — issued by Firebase Auth to browser users. The frontend attaches these automatically via `c2api.ts`. Required for the dashboard to call the C2 API.
2. **Service key** — a static shared secret for internal service calls. The server Discord bot sends this as a `Bearer` token. Set `SERVICE_KEY` in c2-core and `C2_SERVICE_KEY` in the discord-bot to the **same value**.

Admin-only routes (node approve/reject) additionally require the Firebase user to have an `admin: true` custom claim:

```bash
python drb-c2-core/scripts/set_admin.py grant your@email.com
python drb-c2-core/scripts/set_admin.py revoke your@email.com
```

## Discord Bot Token Pool

Edge nodes join Discord voice channels using bot tokens managed by the server. Add tokens in the **Tokens** page of the frontend (or `POST /tokens`). Each token has:
- A friendly name
- The actual Discord bot token string (masked in list views)
- `in_use` flag and the node it's currently assigned to

**Flow when a user runs `/join`:**
1. Discord bot sends a `discord_join` command to c2-core with the target guild and channel IDs.
2. c2-core picks a free token (optionally the one requested via autocomplete), marks it `in_use`, and forwards the token + channel info to the target edge node over MQTT.
3. The edge node starts a `discord.py` bot with that token and joins the voice channel.
4. The bot streams live Icecast audio — only when radio is actively transmitting (speaking ring on = call active).
5. On `/leave`, c2-core releases the token back to the pool.

## Node Lifecycle

1. Edge node starts → sends MQTT `checkin` → appears as **pending** in the dashboard
2. Admin clicks **Approve** → c2-core generates a random API key, stores it in `node_keys`, publishes it to the node over MQTT, updates node to `approved`
3. Admin assigns a radio system via **Config** → c2-core pushes the full system config (frequencies, talkgroups, Icecast settings) over MQTT
4. Edge node writes the config and (re)starts OP25
5. Node sends `online` / `recording` / `unconfigured` status via 30s MQTT heartbeats
6. `node_sweeper` background task marks any node offline after 90s without a heartbeat

## Call & Intelligence Pipeline

This is the full intended data lifecycle for every radio call — from raw RF to searchable, cross-referenced intelligence. The data models and Firestore schema are already designed for this; several pipeline stages are stubs awaiting implementation.

### Call lifecycle (current — fully working)

```
Edge node ──► MQTT call_start ──► c2-core creates CallRecord (status: active)
                                        │  talkgroup_id, talkgroup_name, freq,
                                        │  node_id, system_id, started_at
                                        ▼
                               Firestore "calls" collection
                                        │
                                        ▼
                              Frontend live call feed / map popups
                                        │
Edge node ──► MQTT call_end  ──► c2-core updates CallRecord (status: ended)
                                        │  ended_at, audio_url (GCS link)
                                        ▼
                              Frontend call history / audio playback
```

### Intelligence pipeline (designed — implementation pending)

After a call ends, the following stages should fire in order:

```
CallRecord (ended, audio_url set)
    │
    ▼
[1] TRANSCRIPTION
    Speech-to-text on the GCS audio file (Whisper or Google Speech-to-Text)
    → writes CallRecord.transcript

    │
    ▼
[2] INTELLIGENCE EXTRACTION
    Analyze transcript for:
    - Named entities: unit IDs, street addresses, location references
    - Keywords / keyword sets: fire/EMS/police/hazmat/pursuit/shots fired/etc.
    - Radio codes (10-codes, signals) mapped to plain English
    - Severity scoring
    → writes CallRecord.tags[], CallRecord.location (geocoded if address found)

    │
    ▼
[3] INCIDENT CORRELATION
    Given the extracted entities + tags, decide:
    - Does this call match an existing active IncidentRecord?
      (same location ± radius, overlapping tags, recent time window)
      → link: append CallRecord.call_id to IncidentRecord.call_ids,
              set CallRecord.incident_id, update IncidentRecord.summary
    - Or does this call describe a new event?
      → create IncidentRecord (type, location, title, tags, status: active)
         link the call, set CallRecord.incident_id

    │
    ▼
[4] ALERTS
    If incident is new OR severity exceeds threshold:
    - Trigger configured alert channels (Discord webhook, push notification, etc.)
    - Include: incident type, location, talkgroup, transcript excerpt
```

### Data model fields involved

**`CallRecord`** (Firestore collection: `calls`):

| Field | Type | Populated by |
|---|---|---|
| `call_id` | string | MQTT handler on call_start |
| `node_id` | string | MQTT handler |
| `system_id` | string | MQTT handler (from node's assigned system) |
| `talkgroup_id` | number | MQTT handler |
| `talkgroup_name` | string | MQTT handler |
| `freq` | number | MQTT handler |
| `started_at` | timestamp | MQTT handler |
| `ended_at` | timestamp | MQTT handler on call_end |
| `status` | `active` \| `ended` | MQTT handler |
| `audio_url` | string | Edge node upload → GCS |
| `transcript` | string \| null | **[stub]** Transcription pipeline |
| `incident_id` | string \| null | **[stub]** Incident correlation |
| `location` | `{lat, lng}` \| null | **[stub]** Geocoding from transcript |
| `tags` | string[] | **[stub]** Intelligence extraction |

**`IncidentRecord`** (Firestore collection: `incidents`):

| Field | Type | Description |
|---|---|---|
| `incident_id` | string | UUID |
| `title` | string | Auto-generated or manually set |
| `type` | string | `fire`, `ems`, `police`, `hazmat`, `pursuit`, etc. |
| `status` | `active` \| `resolved` | Updated as calls accumulate or manually resolved |
| `location` | `{lat, lng}` | Geocoded from first call with a location |
| `call_ids` | string[] | All linked CallRecord IDs |
| `started_at` | timestamp | Timestamp of first linked call |
| `updated_at` | timestamp | Updated on each new linked call |
| `summary` | string \| null | Auto-generated from transcripts or manually written |
| `tags` | string[] | Union of tags from all linked calls |

### Backend — what needs to be built

The following do **not exist yet** and need to be created:

- `drb-c2-core/app/routers/incidents.py` — CRUD + manual incident management
- `drb-c2-core/app/routers/alerts.py` — Alert rule configuration
- `drb-c2-core/app/internal/transcription.py` — STT integration (Whisper local or Google Speech API)
- `drb-c2-core/app/internal/intelligence.py` — Keyword/entity extraction, severity scoring
- `drb-c2-core/app/internal/incident_correlator.py` — Match calls to incidents or create new ones
- `drb-c2-core/app/internal/alerter.py` — Dispatch alerts (Discord webhook, etc.)

The `_on_call_end()` handler in `mqtt_handler.py` is the natural trigger point — after updating the CallRecord, it should enqueue the transcription + intelligence pipeline.

### Frontend — what needs to be built

The following pages and nav items are not yet implemented:

- `/incidents` — Incident list and detail view (linked calls, map pin, transcript summary, tags, resolve button)
- `/alerts` — Alert rule configuration (keyword sets, talkgroup filters, notification channels)
- `Nav.tsx` should add **Calls** and **Incidents** as primary nav items; the current Calls page exists but isn't in the design's intended nav hierarchy

The **Map** page should eventually show both node markers and incident pins — incidents with `location` set should appear as color-coded markers (by type/severity) alongside the node status markers.

## Frontend Pages

| Page | URL | Status | Description |
|---|---|---|---|
| Dashboard | `/dashboard` | Working | Live node grid + active call stream |
| Map | `/map` | Working | Leaflet map — nodes color-coded by status, active call popups |
| Calls | `/calls` | Working | Full call history — talkgroup, duration, audio playback, transcript (when populated) |
| Nodes | `/nodes` | Working | Node list; per-node detail for approve/reject/assign system |
| Systems | `/systems` | Working | Create and manage P25/DMR/NBFM radio system configurations |
| Tokens | `/tokens` | Working | Discord bot token pool management |
| Incidents | `/incidents` | **Not built** | Incident list/detail — linked calls, location, summary, tags, resolve |
| Alerts | `/alerts` | **Not built** | Alert rule configuration — keywords, talkgroups, notification channels |

## Makefile Targets

```
make setup          — copy .env.example files to .env
make build          — docker compose build
make up             — docker compose up -d
make down           — docker compose down
make logs           — follow all service logs
make logs-c2        — c2-core logs only
make logs-bot       — discord-bot logs only
make logs-frontend  — frontend logs only
```