init

arch.md

@@ -0,0 +1,76 @@
+# Architecture & Data Flow
+
+## 1. drb-c2-core (The Brain)
+**Purpose:** The central command-and-control server.
+
+**Tech:** Python (FastAPI), MQTT Broker (Mosquitto), Firestore.
+
+**Role:** Manages node health (Heartbeats) and acts as the gatekeeper for configuration commands. It provides the "Raw Data" (Nodes, Frequencies) to the system.
+
+## 2. drb-edge-node (The Ear)
+**Purpose:** The physical radio receiver.
+
+**Tech:** Docker, Python, OP25, Liquidsoap.
+
+**Role:** Listens to RF, demodulates audio, and extracts metadata (Talkgroup, Frequency).
+
+**Check-In:** Sends heartbeat every 30s.
+
+**Output:** Sends Audio and Metadata to Icecast and Metadata to MQTT.
+
+## 3. drb-intelligence-engine (The Analyst)
+**Purpose:** Converts raw audio/text into structured Intelligence.
+
+**Tech:** Python, OpenAI Whisper, LLM (GPT-4o/Claude), Vector Search (Optional).
+
+**Core Logic:** The Incident Aggregator
+Unlike simple logging, this engine maintains "State".
+- **Step 1 Transcribe:** Audio -> Text.
+- **Step 2 Classify:** Determines if the text is "Routine" (Traffic stop, status check) or "Incident" (Fire, Burglary).
+- **Step 3 Correlate (Global Knowledge):**
+  It looks at all active incidents across all nodes.
+  **Scenario:** Node A (PD) hears "Responding to 123 Main St". Node B (FD) hears "Fire at 123 Main St".
+  **Action:** The engine groups both calls into a single Incident document because the location and time window overlap.
+- **Step 4 Update:** Updates the Firestore incidents collection with new summaries, active units, and status.
+
+## 4. drb-web-portal (The Dashboard)
+**Purpose:** User interface for real-time situational awareness.
+
+**Tech:** Next.js, Tailwind, Mapbox/Leaflet.
+
+**Data Model:**
+- **Nodes:** Physical status of receivers.
+- **Incidents:** The "Event" layer. Contains the derived data (Location, Summary, Transcript list).
+
+**Visuals:**
+- **Map:** Shows Incidents, not just radios.
+- **Detail View:** Rolling transcript of a specific event, filtering out unrelated noise from the same radio frequency.
+
+## 5. Data Model (Firestore)
+### Collection: `nodes`
+```json
+{
+  "id": "node-01",
+  "status": "online",
+  "location": { "lat": 0.0, "lng": 0.0 }
+}
+```
+
+### Collection: `incidents`
+```json
+{
+  "id": "inc-timestamp-hash",
+  "type": "Structure Fire",
+  "status": "Active", // or "Cleared"
+  "started_at": "Timestamp",
+  "last_update": "Timestamp",
+  "location": { "lat": 0.0, "lng": 0.0 },
+  "location_history": [
+    { "lat": 0.0, "lng": 0.0, "time": "Timestamp" }
+  ],
+  "units_on_scene": ["E-22", "L-4", "PD-102"],
+  "summary": "Fire reported at 123 Main St. Smoke visible. Units en route.",
+  "transcript_refs": ["Array of IDs pointing to the raw history logs"],
+  "heard_on":["city_fire_department_dispatch_system", "county_police_radio_system"]
+}
+```

overview.md

@@ -0,0 +1,55 @@
+# Project Codex: Distributed SDR Intelligence Network
+
+## 1. Executive Summary
+
+Project Codex is a distributed radio monitoring system that evolves traditional scanning into Real-Time Intelligence. Instead of just listening to audio streams, Codex uses AI to transcribe, analyze, and map public safety radio traffic. It turns a chaotic stream of "Police Radio" into a clean, map-based view of Active Incidents (Fires, Accidents, Chases).
+
+## 2. The Core Workflow
+
+- **Listen:** Distributed Edge Nodes (Raspberry Pis) monitor local frequencies using Software Defined Radios (SDR).
+
+- **Stream:** Audio is sent to a central server; Metadata is sent via MQTT.
+
+- **Analyze:** A central Intelligence Engine transcribes the audio and uses LLMs to understand the context.
+
+- **Correlate:** The system groups related radio calls (even from different departments) into unified Incidents.
+
+- **Visualize:** Users view a Live Map showing active incidents, not just radio towers.
+
+## 3. Key Components
+
+### A. The "Reverse-C2" Control Plane
+
+Edge nodes sit behind firewalls and "phone home" to a central C2 server via MQTT. This allows for centralized management, configuration updates, and health monitoring without complex VPNs.
+
+### B. The Intelligence Layer (Global Knowledge)
+
+This is the brain of the operation. It possesses Global Knowledge of all connected nodes.
+
+- **Incident Tracking:** It detects when a new incident starts and tracks it until units clear the scene.
+
+- **Multi-System Fusion:** If the Police radio and Fire radio both mention the same address, the system recognizes it as one single event.
+
+- **Entity Extraction:** Automatically pulls unit numbers, street addresses, and incident types from the voice audio.
+
+### C. The Web Interface
+
+A Next.js application that provides:
+
+- **Live Map:** A tactical view of what is happening now. Icons appear for active calls and vanish/dim when cleared.
+
+- **Incident Archives:** Detailed pages for every event, showing a timeline of the radio traffic, unit movements, and AI-generated summaries.
+
+- **Fleet Management:** Admin tools to tune radios and manage the edge nodes.
+
+## 4. Development Roadmap Summary
+
+- **Phase 1:** Core Connectivity & Node Health.
+
+- **Phase 2:** Metadata Extraction (getting data out of the radio).
+
+- **Phase 3:** The Web Dashboard & Map.
+
+- **Phase 4:** Audio Routing (Discord/Web).
+
+- **Phase 5:** The Intelligence Engine (AI & Incident Aggregation).

roadmap.md

@@ -0,0 +1,98 @@
+# Project Codex Development Roadmap
+
+## Phase 1: Core Stability & Reliability (Immediate)
+
+Focus: Fixing the "flaky" node connections and ensuring data reliability.
+
+- [ ] Refactor Node MQTT Logic (node_main.py)
+  - [ ] Move initial API check-in out of on_connect blocking callback.
+  - [ ] Implement mqtt_lifecycle_manager with a 30s heartbeat loop.
+  - [ ] Add explicit "Clean Shutdown" signal to the finally block to trigger immediate offline status.
+  - [ ] Verify LWT (Last Will and Testament) triggers only on ungraceful crashes.
+
+- [ ] Update C2 Brain Logic (c2_main.py)
+  - [ ] Update handle_message to refresh last_seen timestamp on any message type.
+  - [ ] Add logic to mark a node as "Unknown/Stale" if last_seen > 90 seconds.
+
+- [ ] Standardize Node Configuration
+  - [ ] Add location object (lat, lng, description) to the node's local config.json.
+  - [ ] Ensure config.json is sent during the initial MQTT Check-in so the C2 knows where the node is geographically.
+
+## Phase 2: The Metadata Pipeline ("The Sidecar")
+
+Focus: Getting real-time talkgroup data out of OP25 and into the cloud.
+
+- [ ] Create Metadata Sidecar (Edge Node)
+  - [ ] Update node_main.py to spawn a subprocess that tails OP25's UDP metadata port or log file.
+  - [ ] Parse lines for: Frequency, Talkgroup ID, Alpha Tag, Signal Strength.
+  - [ ] Publish payload to topic: nodes/{NODE_ID}/metadata.
+
+- [ ] C2 Metadata Ingestion
+  - [ ] Subscribe C2 to nodes/+/metadata.
+  - [ ] Write received metadata to a Firestore Subcollection: nodes/{node_id}/history (for historical playback/logs).
+  - [ ] Update the current_call field on the main node document (for "Now Playing" UI).
+
+## Phase 3: The Web Command Center (Frontend)
+
+Focus: The User Interface for management and monitoring.
+
+Setup
+- [ ] Initialize Next.js 14+ (App Router) project with Tailwind CSS.
+- [ ] Configure Firebase Client SDK (Auth + Firestore).
+
+### Page: Fleet Dashboard (/app/dashboard/page.tsx)
+- [ ] Component: NodeList: Real-time cards for physical radio nodes (Online/Offline status).
+- [ ] Component: SystemLoader: Control panel to push configs to nodes.
+
+### Page: The Live Map (/app/map/page.tsx)
+
+This is the primary intelligence view.
+- [ ] Data Source: Subscribe to incidents collection (not just nodes).
+- [ ] Layer: Incidents
+- [ ] Icons: Dynamic markers based on type (Fire, Police, EMS, Hazard).
+- [ ] Status Indication: Pulse animation for "Active"; Static/Dimmed for "Cleared".
+- [ ] Filter Controls: "Rolling Window" slider (Active Only <-> Active + Past 30 mins <-> Past 24 hours).
+- [ ] Interaction:
+  - [ ] Click Icon -> Preview Popup (Type, 1-line summary, active units).
+  - [ ] Click "More Info" -> Navigates to Incident Detail Page.
+
+### Page: Incident Detail (/app/incidents/[id]/page.tsx)
+
+- [ ] Component: IncidentHeader: Status (Active/Cleared), Duration, Incident Type.
+- [ ] Component: LiveSummary: Streaming text summary of the event so far (generated by LLM).
+- [ ] Component: UnitTracker: List of units currently detected on scene.
+- [ ] Component: TranscriptFeed:
+  - [ ] Filtered list of audio/text clips specifically tagged to this incident_id.
+  - [ ] Support multi-system interleaving (e.g., PD and FD audio mixed chronologically).
+- [ ] Component: LocationHistory: Mini-map showing the incident location (and breadcrumbs if it was mobile/chase).
+
+## Phase 4: Audio & Discord Distribution
+
+- [ ] Persistent Command Bot (Text Interface)
+  - [ ] Create Python bot using discord.py (always online).
+  - [ ] Implement C2 Listener: Subscribes to MQTT discord/commands.
+  - [ ] Feature: !status -> Queries C2 API for active nodes.
+  - [ ] Feature: !join {node_id} -> Spawns/Commands a Voice Bot to join.
+
+- [ ] Voice Streamer Bots (Audio Retransmission)
+  - [ ] Create lightweight bot instances specifically for audio piping.
+  - [ ] Logic: Join voice channel requested by Command Bot.
+  - [ ] Audio: Connect to icecast://central-server/{node_id} using FFmpeg/Opus.
+  - [ ] Presence: Update "Playing" status with current Icecast metadata (Talkgroup/Alpha Tag).
+
+## Phase 5: The Intelligence Engine (Backend)
+
+Focus: Transcribing, Classifying, and Aggregating.
+
+- [ ] Intelligence Service (radio-intelligence-engine)
+- [ ] Module: Transcriber: Whisper STT on "Call End".
+- [ ] Module: Analyzer (LLM):
+  - [ ] Extract Incident Type, Location, Units Involved.
+  - [ ] Generate 1-sentence summary of the transmission.
+- [ ] Module: Incident Aggregator (The "Global Brain"):
+  - [ ] Input: New transcript + Location + Time.
+  - [ ] Logic: Query active incidents within X miles and Y minutes.
+  - [ ] Decision:
+    - Match Found: Append transcript to existing incident_id, update summary, update units, extend "Last Updated" time.
+    - No Match: Create new Incident document in Firestore.
+- [ ] Cross-System Merge: Allow merging transcripts from Node A (Police) and Node B (Fire) into the same Incident if location/context matches.