From 6fdc36fe4c49ae526378cfa7f21155fe969f5107 Mon Sep 17 00:00:00 2001 From: Logan Date: Mon, 6 Apr 2026 02:47:02 -0400 Subject: [PATCH] readme update --- README.md | 304 ++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 215 insertions(+), 89 deletions(-) diff --git a/README.md b/README.md index ac87fe2..f6b2f9a 100644 --- a/README.md +++ b/README.md @@ -1,92 +1,10 @@ # DRB Server -The server-side stack for the Discord Radio Bot system. Handles command-and-control, Firestore sync, Discord slash commands, and the web frontend. +Full-stack backend for the Discord Radio Bot (DRB) system. Receives telemetry from SDR edge nodes over MQTT, stores data in Firestore, orchestrates Discord voice bots through a token pool, and serves a real-time admin dashboard. -## Services +## System Overview -| Service | Description | Port | -|---|---|---| -| `mosquitto` | MQTT broker — receives telemetry from edge nodes | 1883 | -| `c2-core` | FastAPI C2 API — processes MQTT, writes to Firestore, manages nodes/systems/tokens | 8888 | -| `discord-bot` | Discord bot — `/join`, `/leave`, `/status` slash commands | — | -| `frontend` | Next.js admin UI — real-time dashboard, node management, call history | 3000 | - -## Prerequisites - -- Docker + Docker Compose -- A Firebase project with Firestore enabled (Native mode) -- A GCP service account key with Firestore and Firebase Auth permissions -- A Discord bot token (for the server bot that handles slash commands) -- One or more Discord bot tokens in the token pool (for edge nodes to join voice channels) - -## Setup - -```bash -# 1. Copy env files -make setup - -# 2. Fill in secrets -# drb-c2-core/.env — MQTT, Firestore database name, GCS bucket -# drb-server-discord-bot/.env — Discord bot token, C2 URL -# drb-frontend/.env — Firebase config (NEXT_PUBLIC_*), C2 URL - -# 3. Place your GCP service account key -cp /path/to/your-key.json drb-c2-core/gcp-key.json - -# 4. Build and start -make build -make up -``` - -## Environment Variables - -### `drb-c2-core/.env` -| Variable | Description | -|---|---| -| `MQTT_BROKER` | Hostname of the MQTT broker (default: `mosquitto`) | -| `FIRESTORE_DATABASE` | Firestore database name (default: `(default)`) | -| `GCP_CREDENTIALS_PATH` | Path to the GCP key file inside the container | -| `GCS_BUCKET` | GCS bucket name for audio uploads (optional) | - -### `drb-server-discord-bot/.env` -| Variable | Description | -|---|---| -| `DISCORD_TOKEN` | Bot token for the server-side command bot | -| `C2_URL` | Internal URL of c2-core (default: `http://c2-core:8000`) | -| `DEV_GUILD_ID` | Optional guild ID to sync slash commands instantly during dev | - -### `drb-frontend/.env` -| Variable | Description | -|---|---| -| `NEXT_PUBLIC_FIREBASE_*` | Firebase project config (from Firebase console) | -| `NEXT_PUBLIC_FIRESTORE_DATABASE` | Firestore database name (must match c2-core) | -| `NEXT_PUBLIC_C2_URL` | C2 API URL reachable from the browser | - -## Admin Setup - -After the stack is running, grant admin access to your Firebase user: - -```bash -cd drb-c2-core -python scripts/set_admin.py grant your@email.com -``` - -Then sign out and back in to the frontend. - -## Makefile Targets - -``` -make setup — copy .env.example files -make build — docker compose build -make up — docker compose up -d -make down — docker compose down -make logs — follow all logs -make logs-c2 — c2-core logs only -make logs-bot — discord-bot logs only -make logs-frontend — frontend logs only -``` - -## Architecture +DRB is a distributed SDR (Software Defined Radio) monitoring platform. Edge nodes (small Linux machines with RTL-SDR dongles) decode radio systems and stream audio. The server coordinates those nodes, manages which Discord voice channels receive audio, and stores call history. ``` Edge Node (client machine) @@ -98,10 +16,218 @@ Edge Node (client machine) └── discord_join ──► edge node joins Discord voice + streams Icecast Discord User - │ - └── /join /leave /status ──► discord-bot ──► c2-core ──► MQTT ──► edge node + └── /join /leave /status /help ──► discord-bot ──► c2-core ──► MQTT ──► edge node Browser (admin) - └── frontend ──► Firestore (real-time reads) - └── c2-core REST API (writes/commands) + └── frontend ──► Firestore (real-time reads, Firebase Auth) + └── c2-core REST API (writes, commands, token pool) +``` + +## Services + +| Service | Container | Description | Port | +|---|---|---|---| +| `mosquitto` | `eclipse-mosquitto` | MQTT broker — receives telemetry from edge nodes, dispatches commands | 1883 | +| `c2-core` | Python 3.11 / FastAPI | Command & control API — MQTT subscriber, Firestore writes, node/system/call/token management | 8000 | +| `discord-bot` | Python 3.11 / discord.py | Server-side Discord slash command bot — `/join`, `/leave`, `/status`, `/help` | — | +| `frontend` | Node.js / Next.js 14 | Admin web dashboard — real-time node map, call logs, system & token management | 3000 | + +## Directory Structure + +``` +Server/ +├── docker-compose.yml # Orchestrates all four services +├── Makefile # Convenience targets (setup/build/up/down/logs) +│ +├── drb-c2-core/ # FastAPI command & control backend +│ ├── app/ +│ │ ├── main.py # App entry point — router registration, MQTT startup, node sweeper +│ │ ├── config.py # Pydantic settings (reads from .env) +│ │ ├── models.py # Shared Pydantic models (CommandPayload, etc.) +│ │ ├── routers/ +│ │ │ ├── nodes.py # Node CRUD, approve/reject, command dispatch, system assignment +│ │ │ ├── systems.py # Radio system CRUD (P25/DMR/NBFM configs) +│ │ │ ├── calls.py # Call log retrieval +│ │ │ ├── tokens.py # Discord bot token pool — add/delete/list; assign_token/release_token helpers +│ │ │ └── upload.py # Audio file upload endpoint (called by edge nodes) +│ │ └── internal/ +│ │ ├── auth.py # Auth — Firebase ID token OR service key (Bearer) +│ │ ├── firestore.py # Async Firestore wrappers (doc_get, doc_set, doc_update, collection_list) +│ │ ├── mqtt_handler.py # MQTT publisher — commands, config pushes, API key provisioning +│ │ ├── node_sweeper.py # Background task — marks nodes offline after 90s without heartbeat +│ │ └── storage.py # GCS audio file uploads +│ ├── scripts/ +│ │ └── set_admin.py # CLI to grant/revoke Firebase admin custom claim +│ ├── gcp-key.json # GCP service account key — place here, NOT committed to git +│ └── .env.example +│ +├── drb-server-discord-bot/ # Discord slash command bot (server-side) +│ ├── app/ +│ │ ├── main.py # Bot startup, cog loading, guild sync +│ │ ├── config.py # Settings (Discord token, C2 URL, service key) +│ │ ├── commands/ +│ │ │ └── radio.py # /join (with token autocomplete), /leave, /status, /help +│ │ └── internal/ +│ │ └── c2_client.py # Async HTTP client to c2-core — includes service key auth header +│ └── .env.example +│ +└── drb-frontend/ # Next.js 14 admin dashboard + ├── app/ + │ ├── dashboard/ # Overview: active node grid + live call feed + │ ├── map/ # Leaflet map — node locations, status colors, active call popups + │ ├── calls/ # Full call history with duration, audio playback + │ ├── nodes/ # Node list + per-node detail (approve, reject, assign system) + │ ├── systems/ # Radio system CRUD + │ ├── tokens/ # Discord bot token pool management + │ └── login/ # Firebase Auth login page + ├── components/ + │ ├── MapView.tsx # react-leaflet map with status-colored markers + │ ├── NodeCard.tsx # Node status card with action buttons + │ ├── NodeConfigModal.tsx # Modal for assigning a radio system to a node + │ ├── CallRow.tsx # Call history row — talkgroup, duration, audio player + │ ├── AuthProvider.tsx # Firebase Auth context + middleware guard + │ ├── Nav.tsx # Sidebar navigation + │ └── StatusBadge.tsx # Color-coded status pill (online/recording/offline/unconfigured) + ├── lib/ + │ ├── firebase.ts # Firebase JS SDK initialization + │ ├── c2api.ts # Fetch wrapper for c2-core REST API (attaches Firebase ID token) + │ ├── types.ts # Shared TypeScript types (NodeRecord, CallRecord, SystemRecord, etc.) + │ ├── useNodes.ts # Firestore real-time subscription to nodes collection + │ ├── useCalls.ts # Firestore real-time subscription to calls (active + history) + │ └── useSystems.ts # Firestore real-time subscription to systems collection + ├── next.config.mjs # Next.js config — standalone output, transpilePackages for Leaflet + └── .env.example +``` + +## Prerequisites + +- Docker + Docker Compose +- A **Firebase project** with: + - Firestore enabled (Native mode) + - Firebase Authentication enabled (Email/Password provider) + - A GCP service account key with Firestore + Firebase Auth Admin SDK permissions +- A **Discord application** with: + - One bot token for the server command bot (handles `/join`, `/leave`, etc.) + - One or more additional bot tokens for the pool (edge nodes use these to join voice channels) + +## Setup + +```bash +# 1. Copy all env files +make setup + +# 2. Place your GCP service account key +cp /path/to/your-service-account-key.json drb-c2-core/gcp-key.json + +# 3. Fill in secrets — see tables below +nano drb-c2-core/.env +nano drb-server-discord-bot/.env +nano drb-frontend/.env + +# 4. Build and start +make build +make up + +# 5. Grant admin access to your Firebase user +cd drb-c2-core +python scripts/set_admin.py grant your@email.com +# Sign out and back in to the frontend for the claim to take effect +``` + +## Environment Variables + +### `drb-c2-core/.env` + +| Variable | Required | Description | +|---|---|---| +| `MQTT_BROKER` | Yes | Hostname of the MQTT broker (default: `mosquitto`) | +| `MQTT_PORT` | No | MQTT broker port (default: `1883`) | +| `FIRESTORE_DATABASE` | No | Firestore database name (default: `(default)`) | +| `GCP_CREDENTIALS_PATH` | Yes | Path to the service account key inside the container (e.g. `/secrets/gcp-key.json`) | +| `GCS_BUCKET` | No | GCS bucket name for storing call audio recordings | +| `SERVICE_KEY` | Yes* | Shared secret for internal service-to-service auth. Must match `C2_SERVICE_KEY` in the discord-bot env | + +*Required for the server Discord bot to authenticate against the C2 API. + +### `drb-server-discord-bot/.env` + +| Variable | Required | Description | +|---|---|---| +| `DISCORD_TOKEN` | Yes | Bot token for the server-side slash command bot | +| `C2_URL` | Yes | Internal URL of c2-core (e.g. `http://c2-core:8000`) | +| `C2_SERVICE_KEY` | Yes* | Must match `SERVICE_KEY` in drb-c2-core — sent as a Bearer token on all C2 API calls | +| `DEV_GUILD_ID` | No | Discord guild ID to sync slash commands instantly during development | + +### `drb-frontend/.env` + +| Variable | Required | Description | +|---|---|---| +| `NEXT_PUBLIC_FIREBASE_API_KEY` | Yes | Firebase web API key | +| `NEXT_PUBLIC_FIREBASE_AUTH_DOMAIN` | Yes | Firebase auth domain | +| `NEXT_PUBLIC_FIREBASE_PROJECT_ID` | Yes | GCP/Firebase project ID | +| `NEXT_PUBLIC_FIREBASE_STORAGE_BUCKET` | Yes | Firebase storage bucket | +| `NEXT_PUBLIC_FIREBASE_MESSAGING_SENDER_ID` | Yes | Firebase messaging sender ID | +| `NEXT_PUBLIC_FIREBASE_APP_ID` | Yes | Firebase app ID | +| `NEXT_PUBLIC_FIRESTORE_DATABASE` | No | Firestore database name — must match c2-core (default: `(default)`) | +| `NEXT_PUBLIC_C2_URL` | Yes | C2 API URL reachable from the user's browser | + +## Authentication + +Two auth mechanisms are supported on all C2 routes: + +1. **Firebase ID tokens** — issued by Firebase Auth to browser users. The frontend attaches these automatically via `c2api.ts`. Required for the dashboard to call the C2 API. +2. **Service key** — a static shared secret for internal service calls. The server Discord bot sends this as a `Bearer` token. Set `SERVICE_KEY` in c2-core and `C2_SERVICE_KEY` in the discord-bot to the **same value**. + +Admin-only routes (node approve/reject) additionally require the Firebase user to have an `admin: true` custom claim: + +```bash +python drb-c2-core/scripts/set_admin.py grant your@email.com +python drb-c2-core/scripts/set_admin.py revoke your@email.com +``` + +## Discord Bot Token Pool + +Edge nodes join Discord voice channels using bot tokens managed by the server. Add tokens in the **Tokens** page of the frontend (or `POST /tokens`). Each token has: +- A friendly name +- The actual Discord bot token string (masked in list views) +- `in_use` flag and the node it's currently assigned to + +**Flow when a user runs `/join`:** +1. Discord bot sends a `discord_join` command to c2-core with the target guild and channel IDs. +2. c2-core picks a free token (optionally the one requested via autocomplete), marks it `in_use`, and forwards the token + channel info to the target edge node over MQTT. +3. The edge node starts a `discord.py` bot with that token and joins the voice channel. +4. The bot streams live Icecast audio — only when radio is actively transmitting (speaking ring on = call active). +5. On `/leave`, c2-core releases the token back to the pool. + +## Node Lifecycle + +1. Edge node starts → sends MQTT `checkin` → appears as **pending** in the dashboard +2. Admin clicks **Approve** → c2-core generates a random API key, stores it in `node_keys`, publishes it to the node over MQTT, updates node to `approved` +3. Admin assigns a radio system via **Config** → c2-core pushes the full system config (frequencies, talkgroups, Icecast settings) over MQTT +4. Edge node writes the config and (re)starts OP25 +5. Node sends `online` / `recording` / `unconfigured` status via 30s MQTT heartbeats +6. `node_sweeper` background task marks any node offline after 90s without a heartbeat + +## Frontend Pages + +| Page | URL | Description | +|---|---|---| +| Dashboard | `/dashboard` | Live node grid + active call stream | +| Map | `/map` | Leaflet map — nodes color-coded by status, click for active call details | +| Calls | `/calls` | Full call history — talkgroup, duration, audio playback | +| Nodes | `/nodes` | Node list; per-node page for approve/reject/assign system | +| Systems | `/systems` | Create and manage radio system configurations | +| Tokens | `/tokens` | Manage Discord bot token pool | + +## Makefile Targets + +``` +make setup — copy .env.example files to .env +make build — docker compose build +make up — docker compose up -d +make down — docker compose down +make logs — follow all service logs +make logs-c2 — c2-core logs only +make logs-bot — discord-bot logs only +make logs-frontend — frontend logs only ```