126 Commits

Author SHA1 Message Date
Logan 34ca1d0baf Map fixes 2026-05-25 15:28:35 -04:00
Logan 7d6e97fd4a fix: improve geocoding specificity and increase distance threshold for repeater systems
geocode_max_km: 25 → 75 km. The node is a physical receiver, not the system boundary;
digital repeaters extend coverage well beyond 25km (North White Plains at 35.5km from
the Yorktown node is a legitimate Westchester County location).

Query now fully qualified: "High Street" → "High Street, Yorktown, New York".
Added _get_node_state() which reverse-geocodes the node position once (cached) using
Google Maps to get the state name, appended alongside the municipality.
Generic street names (High Street, Main Street) no longer resolve to wrong-country results.
2026-05-25 14:49:02 -04:00
Logan ef8e0d1bfa revert: remove leaflet.gridlayer.googlemutant — incompatible with Next.js 15 bundler
The package consistently throws 'L.GridLayer.GoogleMutant is not a constructor'
due to L-instance conflicts in the webpack bundle, despite multiple workaround
attempts. Removed package, transpilePackages entry, type stub, env var, and all
related component code. Traffic overlay dropped; geocoding (backend) unaffected.
2026-05-25 14:19:21 -04:00
Logan 8a668e6a59 fix: move map action buttons to top-left to avoid legend overlap
Fit and traffic buttons were hidden behind the legend at bottom-right.
Moved both into a column group at top-left below the zoom controls,
where there is clear unobstructed space. Replaced emoji with TRF text label.
2026-05-25 14:09:47 -04:00
Logan dbacd9a9a8 fix: add type stub for leaflet.gridlayer.googlemutant to satisfy TypeScript 2026-05-25 14:02:25 -04:00
Logan a6d841b280 fix: rewrite Google Maps traffic layer to avoid L-instance constructor error
Replace static import + createLayerComponent approach with dynamic import()
inside a useEffect, which ensures leaflet.gridlayer.googlemutant augments the
same L instance that's active at runtime. Add loading=async to Maps JS script.

Traffic is now toggled via a dedicated button (green when active) rather than
LayersControl, bypassing the react-leaflet layer lifecycle that caused the
constructor conflict.
2026-05-25 13:59:47 -04:00
Logan 96bba45ffa fix: correct npm package name to leaflet.gridlayer.googlemutant 2026-05-25 13:44:31 -04:00
Logan 6a9fe5d26f feat: replace Google tile URL hack with leaflet-google-mutant for traffic layer
Add leaflet-google-mutant@0.16.0 (exact/locked version) as a proper bridge
between the Google Maps JavaScript API and Leaflet. The old mt{s}.google.com
tile URL approach was unofficial and produced empty tiles.

Traffic layer now renders via createLayerComponent + googleMutant, loaded only
after the Maps JS API script is injected and ready (keyed off NEXT_PUBLIC_GOOGLE_MAPS_API_KEY).
Added leaflet-google-mutant to transpilePackages in next.config.mjs.
2026-05-25 13:41:10 -04:00
Logan 0279a82b10 feat: replace Nominatim geocoding with Google Maps API; add TOC map improvements
Switch geocoding from Nominatim to Google Maps Geocoding API for accurate
local place name resolution (bounds-biased, with 25km distance rejection guard).
Remove the now-unused _get_node_place reverse-geocoder and _node_place_cache.

Map page (TOC improvements):
- Weather radar tiles auto-refresh every 5 minutes via radarEpoch key cycling
- Google Maps traffic overlay added to LayersControl
- Live 24h clock overlay at bottom-left for situational awareness
- Incident sidebar cards now show age (time since dispatch) and unit count
2026-05-25 13:27:19 -04:00
Logan 0db09d6bf7 fix: reject geocode results outside node jurisdiction
Nominatim's viewbox is advisory (bounded=0), so ambiguous place names like
"Pinebrook" can resolve to locations 30-40km away in the wrong town. Added
a post-geocode distance gate: results farther than geocode_max_km (default
25km) from the node are discarded with a warning log rather than written to
the incident.

Also logs distance on successful geocodes for easier audit.

New config setting: geocode_max_km (float, default 25.0)
2026-05-25 13:09:10 -04:00
Logan 4b7d9dd49a feat: enrich correlation debug with fit_signal and orphan breakdown
_call_fits_incident now returns (bool, signal_str) so each correlation
decision records exactly what evidence fired: unit_overlap, vehicle_overlap,
location_proximity, time_fallback, tactical_default, or the corresponding
false-return variants (unit_loc_conflict, content_divergence, etc.).

- corr_fit_signal and corr_matched_units written to call docs for
  fast/single and fast/disambig paths
- Admin debug endpoint exposes the new fields in calls_detail
- Orphan section adds orphans_by_talkgroup summary (count, no-type count,
  sweep-exhausted count per TGID) and raises orphan limit 100 → 250
- Admin page shows corr_path and fit_signal distribution panels above raw
  JSON; time_fallback highlighted in yellow as a diagnostic marker

No correlation logic changed — diagnostic data only.
2026-05-25 12:54:34 -04:00
Logan 7dd090e8b2 fix: raise garbage-transcript threshold to avoid false positives on plate reads
Phonetic run threshold 5 → 12: a plate spellout ("Foxtrot Alpha Uniform Lima
Kilo...") produces 6–8 consecutive phonetic words, triggering false positives
and blocking intelligence extraction on legitimate calls. 12 is safely above
any real spellout (~8 max) while still catching the full-alphabet hallucination
(26 words). Also writes skip_reason="garbage_transcript" to the call doc and
surfaces it in the admin correlation debug endpoint.
2026-05-25 03:31:43 -04:00
Logan 92c9d8effc fix: garbage transcript detection, county geocoding, dispatch channel detection
- intelligence.py: detect Whisper phonetic-alphabet hallucinations before
  sending to GPT; skip extraction entirely to prevent fake units/tags
  corrupting correlation
- intelligence.py: upgrade node reverse-geocode from zoom=5 (state) to
  zoom=10 (county) and include county in address queries so common street
  names (e.g. "East Main Street") resolve to the correct county
- incident_correlator.py: add "patched" and "primary" to dispatch channel
  regex so patched trunking channels are treated as shared backbones
- incident_correlator.py: add 20-min idle gate for tactical channel default
  so a reused frequency can't absorb a new unrelated incident
2026-05-24 01:30:40 -04:00
Logan 1071bcd3e8 fix: map overlay clicks, layer overlap, fan spacing, geocoding radius
- Move incident panel to left side (was topright, conflicting with LayersControl)
- Move legend to bottom-right, raise auto-fit button to clear it
- Tighten fan card step 7→5px for closer grouping
- Geocoding: remove bounded=1 hard clip, widen bias radius 0.1°→0.5° (~55km)
  so addresses like "34 Carlton Drive" resolve outside the node's immediate area
2026-05-24 00:20:11 -04:00
Logan 4fc44dcc86 feat: map overhaul, kiosk mode, RR importer, duplicate system
Map (MapView.tsx):
- Fan/hand-of-cards marker clustering: groups nearby markers by pixel
  proximity (union-find), renders as rotated color cards showing all types
- Pulsing ring CSS animation on recording nodes (pulse-ring keyframe)
- Live incident overlay panel — right sidebar (desktop) / bottom drawer (mobile),
  clickable to flyTo incident location
- Auto-fit button (⤢) fits all markers in view with fitBounds
- "Live · Xs ago" timestamp badge (refreshes every 10s)
- Weather Radar layer (NEXRAD via Iowa Env Mesonet, no API key)
- ADS-B + Meshtastic placeholder layers (off by default)

Map page (map/page.tsx):
- Fullscreen / kiosk toggle: fixed z-50 overlay covers nav, map fills viewport
- lastUpdated tracking passed to MapView for Live timestamp

Systems page (systems/page.tsx):
- Duplicate System button: opens form pre-filled with Copy of <name>
- RadioReference HTML import: file upload → DOMParser validates .rrlblue
  structure, parses talkgroup categories, modal lets user select which
  categories to import, auto-maps RR tags to local tags (law→police, etc.)
2026-05-23 23:52:49 -04:00
Logan 6397e24035 Correlation updates 2026-05-23 22:55:50 -04:00
Logan 5a18a66d77 fix ppm bug 2026-05-23 18:22:47 -04:00
Logan 35ce8e911e audio fixes attempt 2026-05-23 14:59:51 -04:00
Logan 9cf8fd4221 fix date filter 2026-05-23 13:28:23 -04:00
Logan 0ceb0227c8 call row fix 2026-05-23 13:11:42 -04:00
Logan fc993fdfe6 call table update 2026-05-23 13:05:53 -04:00
Logan 9d73fc52fa STT bugfix 2026-05-17 19:37:38 -04:00
Logan 97ed691cd2 correlation upgrades 2026-05-17 19:05:52 -04:00
Logan bcc3d3406d add debug in admin 2026-05-17 18:42:42 -04:00
Logan 4006232c85 Filter calls in ui 2026-05-10 22:17:20 -04:00
Logan 4c3b1fcc84 UI Updates 2026-05-10 21:47:34 -04:00
Logan 8b660d8e10 feat: incident correlation overhaul, signal-based auto-resolve, token fixes
Correlator
- Raise fast-path idle gate 30 → 90 min (tg_fast_path_idle_minutes)
- Fix disambiguate always-commits bug: run _call_fits_incident on winner
  before committing; fall through to new-incident creation if it fails
- Add unit-continuity path (path 1.5): matches all_active by shared unit
  IDs with a reassignment guard, bridges calls past the idle gate
- Add tag-based incident_type inference (_TAG_TYPE_HINTS) as GPT fallback,
  rescuing tagged calls that would have been dropped (616 observed orphans)
- Add master/child incident model: _create_master_incident, _demote_to_child,
  _add_child_to_master; new incidents stamped incident_type="master"
- Add cross-system parent detection (_find_cross_system_parent): two-signal
  scoring (road overlap=0.4, embedding≥0.78=0.3, proximity=0.3, threshold=0.5)
  wired into create-if-new path; creates master shell on first cross-system match
- Add maybe_resolve_parent: auto-resolves master when all children close;
  called from upload pipeline (LLM closure) and summarizer stale sweep
- Add signal-based auto-resolve via units_active/units_cleared tracking:
  GPT now extracts cleared_units per scene; _update_incident moves units
  between active/cleared lists and resolves the incident when active empties;
  stored on call doc for re-correlation sweep reuse
- Add _create_incident initialization of units_active/units_cleared fields

Re-correlation sweep
- Add corr_sweep_count + MAX_SWEEP_ATTEMPTS=3: orphans get 3 attempts
  then are tombstoned as corr_path="unlinked", ending the re-sweep loop
  (previously hammering each orphan 29-31 times per shift)

Intelligence extraction
- Add cleared_units to GPT prompt schema and rules
- Extract and propagate cleared_units per scene; merge across scenes;
  store on call doc for re-correlation sweep

Token management
- Fix token release bug: remove release_token call on discord_connected=False
  in MQTT checkin (transient Discord drops were orphaning bots mid-shift)
- Add PUT /tokens/{id}/prefer/{system_id} endpoint: lock a bot token to a
  system; pass _none as system_id to clear; stored bidirectionally on both
  token and system documents
- discord_join handler resolves preferred_token_id from system doc and passes
  system_name in MQTT payload
2026-05-10 19:49:05 -04:00
Logan 7e1b01a275 Updates to reduce firestore calls to try and stay in free tier
### Firestore read reductions

**1. `doc_get_cached()` in `firestore.py` — new 5-min TTL cache**
One place, benefits everything. System and node config documents almost never change during a monitoring session.

**2. System doc: 4 reads → 1 per call**
| Before | After |
|---|---|
| `upload.py` — `doc_get("systems")` for ai_flags | `doc_get_cached` |
| `transcription.py` — `get_vocabulary()` → `doc_get("systems")` | cache hit |
| `intelligence.py` — `get_vocabulary()` → `doc_get("systems")` | cache hit |
| `intelligence.py` — `doc_get("systems")` again for ten_codes | eliminated (reads same cached doc) |

**3. Node doc: cached in `_on_call_start` and `intelligence.py`**
The node is read every call event to get `assigned_system_id` and lat/lon for geocoding. Both now use the cache — node assignments and positions essentially never change at runtime.

**4. Node sweeper: 30s → 90s interval**
The sweeper was doing a full node collection scan 3× more often than necessary — the offline threshold is already 90s. Cuts sweeper reads by 66%.

**5. Vocabulary induction: scans all-time calls → last 7 days**
Previously fetched every ended call for a system (could be thousands). Now scoped to the last 7 days.

> **Note:** The vocabulary induction query `(system_id == X, ended_at >= cutoff)` needs a Firestore
> composite index on `(system_id ASC, ended_at ASC)`. When the induction loop first fires it will log
> an error with a Firebase Console link to create it in one click.
2026-05-04 02:05:00 -04:00
Logan 97f4286810 Add debugging 2026-05-04 01:46:56 -04:00
Logan e704df1a62 # app/internal/incident_correlator.py
- *`correlate_call`* — added units and vehicles optional params; when provided (per-scene from intelligence extraction), they take priority over the merged call-document values, preventing multi-scene unit contamination
- *Cross-TGID correlation path (2.5)* — *new path between location and slow paths*: when a call shares 2+ unit IDs with a recent same-system, same-type incident AND embedding similarity ≥ 0.85, it links them — catches multi-talkgroup pursuits like the bicycle search that split across dispatch/tactical/geographic channels
# `app/internal/intelligence.py`
- *`reassignment` field* — added to the GPT-4o-mini prompt schema and rules; `true` when dispatch is actively pulling a unit to a new, different call (not a status update or en route acknowledgement); returned in every processed scene dict
- *Tag location rule* — added explicit instruction to the prompt: tags must describe what happened, not where; place names, road names, and talkgroup names are explicitly forbidden as tags
# `app/routers/upload.py`
- Both scene correlation call sites (`_run_extraction_pipeline` and `_run_intelligence_pipeline`) now pass `units=corr_units` where `corr_units = [] if scene.get("reassignment") else scene.get("units") `— suppresses unit overlap matching when a unit is being reassigned to a new call, preventing chaining into their previous incident
- Both sites also pass `vehicles=scene.get("vehicles")` (per-scene vehicles, from the multi-scene units fix)
# `app/config.py`
- `embedding_cross_tg_threshold: float = 0.85` — threshold for the new cross-TGID path
2026-05-04 01:33:03 -04:00
Logan f6897566f8 Fix tags, titles, and hallucinations 2026-05-04 01:13:18 -04:00
Logan 531ce64eeb Fix system AI flag bug 2026-04-27 00:58:05 -04:00
Logan f8a9cda27e update firestore to FieldFilter 2026-04-27 00:54:35 -04:00
Logan 640667c9f9 Implement per-system AI flags 2026-04-27 00:50:01 -04:00
Logan 5f83194420 Build fix 2026-04-27 00:40:40 -04:00
Logan c959437059 Implement Admin UI to disable AI components 2026-04-27 00:37:51 -04:00
Logan 92c8351864 Correlation updates 2026-04-26 11:01:32 -04:00
Logan 64232279ca fix calls 2026-04-26 00:04:32 -04:00
Logan 317f9d2a9d Updates to intel and correlation 2026-04-23 01:26:41 -04:00
Logan bcd3406ae8 Make calls playable in the same window 2026-04-21 22:44:38 -04:00
Logan e70e7c0be9 Use UV for pip 2026-04-21 22:36:01 -04:00
Logan 88103c8011 UI Fix 2026-04-21 22:26:33 -04:00
Logan 65839a3191 Implement recorrelation logic 2026-04-21 22:19:57 -04:00
Logan 338b946ba3 Start to learn vocab from talkgroups to improve accuracy of STT 2026-04-21 22:17:30 -04:00
Logan 6612e4b683 Big updates 2026-04-21 01:51:23 -04:00
Logan 788afca339 Update geocoding intel 2026-04-19 23:27:51 -04:00
Logan 0fb97c3b3c fix corrections ui 2026-04-19 23:01:28 -04:00
Logan ba43796c51 Updates, big updates
incident_correlator.py — full rewrite: always runs on every call, fetches all active incidents cross-type, fast path collects all talkgroup matches and disambiguates by unit/vehicle overlap → location proximity → embedding, new location proximity path, slow path requires location corroboration, "Auto:" stripped from titles, "auto-generated" tag added, units/vehicles now accumulated on update
intelligence.py — resolved field in GPT schema, returned as 5th value
upload.py — both pipelines unpack 5-tuple, always call correlate, auto-resolve on resolved=True
summarizer.py — stale sweep runs each tick, resolves incidents idle for 90+ minutes
config.py — correlation_window_hours=2, embedding_similarity_threshold=0.93, location_proximity_km=0.5, incident_auto_resolve_minutes=90
2026-04-19 22:53:53 -04:00
Logan f9d4fcbc39 ui fix 2026-04-19 17:07:40 -04:00
Logan 303c5b13cf big ui and intel updates 2026-04-19 16:48:55 -04:00