QA (drb-qa-review) blockers: sdr_count defaulted to 1 for nodes that
never sent it, so the panel claimed 'reports 1 SDR (0 spare)'. It is now
None until reported, and the count is only quoted alongside a real
secondary_sdr_running report. Rows read 'Not reported' instead of
'Waiting for SDR' when the node hasn't said what's running (closes#187).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pairs with node-26 feat/secondary-sdr-priority. NodeRecord gains
secondary_sdr_priority (ordered; SDRs beyond OP25's run it top-down) and
secondary_sdr_running, both mirrored from the node's checkin.
PATCH /nodes/{id} accepts the priority, validates it, and sends it as a
'set_secondary_priority' MQTT command. A priority-only change never
re-pushes system config, because that restarts OP25. The node detail page
gets a 'Secondary SDRs' section (admin-editable) with enable, reorder,
save, and live Running / Waiting-for-SDR state from the checkin.
Verified: c2-core pytest 482 passed; frontend tsc --noEmit clean.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
QA follow-ups on the ADS-B/AIS overlays:
- Legend gains an aircraft altitude key (tar1090 ramp) while the Aircraft
overlay is on (closes#185).
- AIS aids to navigation (MMSI 99xxxxxxx) render as small yellow buoy
diamonds labelled 'Aid to navigation', no speed; vessels get a 22px
outlined hull, and heading 511/360 ('not available') no longer
rotates the icon (closes#186).
- Aircraft details panel becomes a bottom sheet above the incident drawer
below md, instead of fighting the layers control at the top right.
Verified: tsc --noEmit clean (node:20 on radio-box).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The popup sat on top of the plane and hid the trail it had just drawn.
Details now render in a panel docked top-right (portaled into the
Leaflet container, click/scroll propagation disabled), so the map can
be panned to follow the path. Clicking empty map or the same plane
deselects; hover tooltip unchanged.
Verified: tsc --noEmit clean (node:20 on radio-box).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Icons were 16px accent-colored glyphs, indistinguishable from OSM's own
airport symbols. Now a 30px outlined airliner silhouette filled on
tar1090/ADS-B Exchange's altitude hue ramp, with a callsign/altitude
hover tooltip; the selected aircraft grows and gets a white outline.
Clicking an aircraft draws the path heard so far, segment-colored by
altitude. c2-core writes one point per position change to
aircraft/{icao}/positions (deduped in-process, writes now concurrent);
points carry expire_at and a TTL fieldOverride deletes them after ~24h.
Trail reads are gated on the parent aircraft doc's org via get(), so the
query needs no org filter or composite index. The latest stretch without
a 20-min gap counts as the current flight.
Verified: c2-core pytest 479 passed; frontend tsc --noEmit clean (node:20
container on radio-box).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The deploy-firestore-rules job has failed on every push because
FIREBASE_TOKEN was never set, so rule changes (e.g. aircraft/vessels for
node-26#9) never reached prod. Switch to a dedicated least-privilege
service account whose JSON key lives in FIREBASE_SA_KEY; login:ci tokens
are deprecated and carry their minter's full access. Key is written to
RUNNER_TEMP at 0600 and removed on exit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A 09-22 replay stop was titled "Traffic Stop at 96 times 5" — a disposition
code read aloud, extracted as the location (server-26#170). clean_location
now rejects "N times N", ten-codes, "signal N", "code N", "condition N".
c2-core: 476 pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner: plate reads should become traffic stops. Held-out replay of 09-21
(server-26#170): Ossining's Post 4 stops were read out only as plates
("Frank David Boy, 4514", "Lincoln, Charlie, Robert, 7-4-0-7") and never
became incidents.
The self-initiated backstop now treats two+ phonetic letters followed by
3-7 digits as a stop — but only when extraction found no other event in
the call (a plate on an MVA, tow or parked-car complaint stays with that
event), and never on MTA/rail/bridge/fire/EMS/DPW talkgroups.
c2-core: 475 pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A day of replay runs (server-26#170) spent ~$5 of Gemini on ~7 two-hour
windows (~$0.70 per 290 calls) — several dollars a day per live deployment
for correlation alone — and nothing could say where it went (#45). Gemini
3.x thinks by default and bills it as output; the deprecated
google-generativeai SDK these calls used cannot set a thinking level.
- app/internal/gemini.py: every Gemini call (correlation + transcript
correction) goes through google-genai with JSON mode, an explicit
thinking level, and logs in/out/thinking tokens. A model that rejects
the level is retried without it once and remembered, so the tier is
never lost to a config param. API failures still raise for ai_health.
- correlator: thinking_level "minimal" (a link/new/orphan choice).
transcript correction: "low" until a replay shows minimal is safe.
- replay: runs record real Gemini token usage (metrics.gemini_usage),
shown in the Replay tab.
- requirements: google-genai.
c2-core: 474 pass. Frontend typecheck not run (no Node on this box).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replay of 433b35d (server-26#170): traffic stops now open incidents
("45 Adam" stop, "CM2" stop), but the bridge MVA split 33/56 — one
transmission ("transport complete") was read as scene-resolved at 14:44
and an LLM closure was final, so the rest of the MVA opened a new one.
- LLM closure is now provisional (reopenable), like a timer close: it is
inferred from a single transmission.
- review of 433b35d: backstop only matches a unit's own "on a stop"
self-report (bare "car stop" mentions and "pull over" dropped), never on
MTA/rail/bridge/fire/EMS/DPW talkgroups ("Train 4 holding on the stop"),
negation looks 5 words back, and <=5-word reports ("Adam 3 on a stop")
get a minimal scene instead of being skipped before the backstop.
c2-core: 471 pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner: traffic stops should show on the portal — otherwise they are only
visible in the archive. In the 09-22 replay (server-26#170) every Ch 1
stop ("45 Adam on a stop, Eastbound Central Express", "CM2 on the stop,
southbound") came back from extraction untyped, untagged and routine —
read as status traffic after #138 — so the creation gate never opened one.
- prompt: a unit reporting its own activity (on a stop, out with a vehicle
or pedestrian) is a real event: police, tagged, at least minor; the plate
lookups for it belong to it.
- deterministic backstop after extraction: stop / "put me out with"
phrasing adds a "traffic-stop" / "self-initiated" tag (the substance the
creation gate counts), police type if none, minor if routine. Negated
phrasing ("not pull the car over") is left alone; nothing is downgraded.
c2-core: 469 pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- only a substantive call after the close reopens a timer-closed incident;
a thin "10-4" (doesn't refresh updated_at) or a sweep link of a call from
before the close rides along without reopening — otherwise the next
sweep closed it again and the portal flickered.
- substantive_call_count counts a call once, not once per scene.
- a timer-closed incident is never adopted as a cross-system parent.
Replay e972cac (correlation-only on 731b54b's extraction) vs the 09-22
answer key: pairwise F1 0.548 -> 0.807 (precision 0.839 -> 0.956, recall
0.407 -> 0.698); the bridge MVA is one 76-call incident instead of two
40-call halves. server-26#170.
c2-core: 468 pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hand-labelling the 09-22 10:00-12:00 ET replay window (server-26#170,
answer key replay_groundtruth_0922.json) found ~25 real incidents, of which
only ~5 had an audible clear — most jobs clear by MDT, so the quiet timer is
the close for most incidents and a flat 90 minutes left a lockout or a plate
check "active" on the portal an hour after it ended.
- summarizer: timer close after 30 min quiet for routine/minor, 60 moderate,
90 major/unknown. A timer close is provisional: reopenable=True.
- correlator: reopenable incidents inside incident_reopen_window_minutes
(90, since last substantive call) stay candidates; linking a call to one
reopens it (status active, reopened_count++). The sweep expires the flag
so the reopenable pool stays bounded. Real clears (units_cleared,
llm_closure) are never reopenable.
- cap: incident_max_calls counts substantive calls only
(substantive_call_count). The bridge MVA hit 40 in 32 min with ~40% thin
replies, split in half, and the second half took another job's title.
c2-core: 467 pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of the first clearance fix found it pushes toward early resolve:
- parser cleared units on questions ("45-9, are you clear?"), negations
("not clear yet"), orders ("clear the scene", "clear to transport"),
places/times ("Room 2 clear", "1400 hours, clear") and split "45 9" into
unit 45. Now rejects "?", not/is/are/you, anything after the status word
but sign-offs, place/time words; joins "45 9" -> "45-9".
- a clear from a unit never active on an incident was recorded in
units_cleared and could pass the all-clear gate. Only units actually
active there can clear there now.
- clearance-only calls skip the LLM tier (same as thin calls): only the
rules engine's unit match can say which incident a 10-8 belongs to.
- replay incident view carries srcaddr/srcaddrs for the radio-ID clearance
investigation.
Replay 09-22 10:00-12:00 ET with f0a88d4: real clears 0 -> 2 (both LLM
closure), unit clears still 0 — the parsed clears are right but those
units were never recorded as assigned (Whisper mangles unit IDs at
dispatch), which this commit does not fix.
c2-core: 465 pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replay of 09-22 10:00-12:00 ET (server-26#170): 0 of 19 incidents resolved
on a clear, 19 on the idle timer, although 25 transmissions said 10-8/clear.
Three independent breaks:
1. Short clears never reached extraction. "45-9, I'm clear." is <=5 words,
so extract_scenes skipped it before GPT and cleared_units stayed empty.
A rule parser now names the unit when it precedes the status word
(never guesses: "10-8, 10-8." / "CMT clear." clear nobody) and returns a
minimal scene that links by unit overlap but cannot open an incident.
2. Clearance compared unit strings exactly, so "11-Adam" clearing never
removed "11 Adam". Now by _normalize_unit key.
3. units_active collected "Desk", "Central", "Division", "unknown", plate
phonetics — none of which ever clear, so all-clear could never pass.
Only units carrying a number (and not a ten-code) are tracked now; the
rest stay in `units` for matching.
c2-core: 463 pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first replay (server-26#170) aborted on 5x 'model is unavailable'
from the tiebreak tier. Google's model list shows 2.5 closed to new
projects and no stable Pro model; 3.8-flash is the newest stable.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
First replay (290 calls, 09-22 10:00-12:00 ET) produced 0 incidents and
no errors: every gpt-4o-mini extraction failed and _sync_extract
swallowed it as "no scenes". Same shape as #169 — and the live extraction
tier in /health/ai had no reporter at all, so this has been invisible in
production too.
- intelligence: API failures propagate out of _sync_extract; extract_scenes
reports them to ai_health ("extraction" tier, billing/dead-model
classified) and still returns [] so the pipeline degrades as before.
- ai_health: inside a replay sandbox, failures go to the run's own sink
instead of being dropped.
- replay: aborts after 5 permanent failures on a tier, naming the cause;
run metrics carry ai_failures; UI shows them.
- replay estimate: audio minutes from started_at/ended_at (no duration
field exists on call docs).
- ReplayTab exposes the loaded run on window.__drbReplay for in-page
analysis.
c2-core: 458 pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Correlation has only ever been measured through live AI windows: days of
wall time per change, and the 09-20→22 window was invalidated outright by
unfunded AI accounts (#169). Recordings are kept regardless of AI, so the
traffic to measure against already exists.
- internal/replay.py: runs a time range of real calls through the live
pipeline code in original order, clock pinned per call, into
replay_runs/{run_id}/calls|incidents. Modes: audio (re-transcribe),
transcripts (re-extract), reuse (correlation only from a prior run's
scenes). Simulates the idle-resolve and orphan-recorrelation sweeps on
virtual time. No alerts, summaries, vocab, AI-health alerts or pending
terms. One run at a time, <=5000 calls, <=7 days.
- firestore.py: ContextVar sandbox redirect for calls/incidents.
- clock.py: ContextVar-pinnable now(), used on the correlation path.
- feature_flags.py: ContextVar flag override so replay runs with live AI off.
- upload.py: scene loop extracted to _extract_and_correlate, shared by the
live pipeline and replay so replay measures the code that runs live.
- resolved_via on every incident resolve, so a real clear can be told
from the idle timeout — live and in replay.
- routers/replay.py + /admin Replay tab: estimate, start, compare runs,
drill into incidents with audio.
Reviewed by drb-correlation-review; its leak and fidelity findings are
fixed and covered by tests. c2-core: 456 pass. Frontend typecheck not run
(no Node on the authoring box).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Incident started_at is an isoformat() string (incident_correlator.py,
routers/incidents.py), not a Firestore timestamp like calls. The range
bounds were Dates, which Firestore compares by type, so any date range
returned zero incidents. Bounds are now UTC ISO strings in the same
"+00:00" shape, which order lexicographically by time.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/calls/search scans a 200-row window and returns 50, but the next cursor
was always the last SCANNED row — so with an unfiltered list each page
jumped past the 150 matches it had already read and not shown. Resume
after the last RETURNED row when matches overflow the page; keep the
last-scanned cursor only when the page holds every match (the sparse-
filter case that cursor exists for). Same fix for /calls/eval-queue.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Incidents and Archive get a from/to date range (native date inputs,
local-day bounds). Incidents filters in the Firestore query; Archive
passes date_from/date_to to GET /calls/search, which applies them as a
started_at range — both ride the existing org_id/started_at index.
Also fixes /calls/search and /calls/eval-queue paging: the cursor went to
Firestore as a raw ISO string against a timestamp field, which compares
by type rather than time, so "Load more" re-read the first page. Cursor
and range bounds are now parsed to datetimes (400 on garbage).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Incidents page: text search (title, location, summary, units, vehicles,
tags, location mentions), status and type filters, and a Load more button
that pages the Firestore query 100 at a time. Filtering runs over the
loaded window, and the page says so when older incidents exist.
Archive (/calls): readable by every org member, not just admins.
GET /calls/search now takes any Firebase token scoped to the caller's org
— the Firestore rules already let members read every call in their org,
so this widens nothing. Attach/detach stays admin-only (UI and routes).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Backend: three new routes on the calls router, deliberately separate from
PATCH /{call_id}/transcript (a production correction with real side effects
-- re-extraction, incident unlinking, vocabulary learning). This is pure
measurement and must never share that code path.
GET /calls/eval-queue -- calls with a transcript but no
eval_transcript yet, paged (same bounded-
window-plus-cursor shape as /search)
PUT /{call_id}/eval-transcript -- records eval_transcript/_by/_at only;
never touches transcript/transcript_corrected
GET /calls/eval-stats -- eval_count + average word error rate of
the raw and corrected machine transcripts
against the human-verified ones
internal/wer.py: standard word-level Levenshtein WER. Returns None (not 0.0)
when the reference is empty -- a call nobody transcribed must not score as a
perfect match.
Frontend: a new "STT Eval" tab on /admin -- one call at a time, audio player,
a textarea pre-filled with the machine transcript to correct into ground
truth, Save & next / Skip, running WER stats at the top. Built for working a
handful of calls at a time over however many sittings it takes, not a
one-shot form: the queue auto-refills from where the last save left off.
Verified: 438 pass, 0 fail (12 new backend tests). Frontend is UNVERIFIED --
this box has no Node.js/npm (confirmed absent), so neither typecheck nor the
dev server could be run. Matches existing code patterns and the CallRecord/
c2api types by manual review only.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
correct()'s prompt says "Do NOT expand ten-codes" and "NEVER add
information", but nothing checked the model's output against its own
rules -- raw["corrected"] was accepted verbatim past a non-empty/changed
check, and the "changed" field it returns was logged for audit and never
validated.
Caught live: the same call's raw vs corrected transcript showed "10-7"
rewritten to "10-13" in one place and "10-4" in another, plus a bare "7"
expanded into "ShotSpotter" -- alongside genuinely good fixes (Holmes
Street and 4th and Rowe -> Home Street and Forest Ave, from this system's
own vocabulary). A wrong 10-13 standing in for a real 10-7 reads exactly
as trustworthy as a correct transcript, which is worse than leaving the
raw mishearing in place.
_code_tokens() extracts every ten-code/signal-shaped token from the
original and corrected text/segments; any change to that set discards
the correction and falls back to raw. Checked independently for the
joined text and for segments, consistent with the existing
all-or-nothing segment-alignment rule.
Does not catch a wrong word swapped for another equally plausible
non-code word -- that class still depends entirely on the model
following its own prompt. Filed server-26#161 for the broader STT/audio
quality initiative this belongs alongside.
Verified: 426 pass, 0 fail (4 new tests).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
cat file 2>/dev/null || echo latest only falls back when cat itself fails
(nonzero exit, i.e. the file is missing) -- a file that EXISTS but is EMPTY
makes cat succeed with empty output, so PREV_TAG became "" instead of
"latest". That "" failed the emptiness check further down and exited 1 --
AFTER git pull + docker compose up -d had already succeeded. Worse: exiting
there skips the Health check step (Gitea Actions doesn't run later steps
after a failure), and Health check is the ONLY step that ever writes a real
value to .last_good_tag. Self-perpetuating: once the file went empty, every
future deploy failed the same way forever, with the app itself deploying
fine underneath it every time (confirmed against run 611: deploy log shows
all three images pulled/recreated/started at 66bbf5b, and /health
self-reports that exact GIT_SHA, baked into the image at build time --
two independent signals, same commit).
${VAR:-default} covers empty and unset in one expansion, matching the
fallback behavior the surrounding comment already documented as intended.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
_geocode_location's node-distance fallback (geocode_max_km, 40km) is a
proxy for "is this plausible" that only makes sense when the node's own
position is the best guess we have at the area. It doesn't hold for a
citywide/patched feed: node-002 sits in Westchester but relays "New York
City - NYPD Citywide 2 Patch", ~56km from the addresses on it. Every real,
correctly-geocoded address on that talkgroup was rejected by this check,
every time -- location_coords stayed permanently null for the whole
system, which killed the location_proximity correlation signal and let
duplicate incidents form for the same event reported at two nearby
addresses two minutes apart ("Shots Fired at Jackson Avenue" /
"Shots Fired at 1108 Jackson Avenue", 2026-09-20 ~23:00 UTC, merged by
hand via the Archive page's attach/detach while this fix went in).
trust_named_region skips the node-distance rejection exactly when the
query already carries a place name that isn't the node's own position --
operator-set area_context, or a municipality parsed from the talkgroup's
own name. The anchor path is untouched; an anchor's own radius is always
authoritative when one has been resolved.
Also fixes a compounding defect found while verifying: the query was
grafting the node's own COUNTY onto an already-self-named region
("...New York City..., Westchester, New York"), which is self-contradictory
and could degrade the geocode independent of the distance check. The
node's county is now used only when nothing else names the place; state
stays in both branches since it's coarse enough to be correct either way.
Extracted the query-assembly logic into a pure, unit-tested helper
(_location_query_parts) rather than testing it only through the full
extraction pipeline.
Filed #160 as a follow-up: place_verifier.py's verify() has the identical
no-anchor gap for transcript place-name correction, not fixed here.
Verified: 422 pass, 0 fail (9 new tests).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The fast/thin path (short acks like "10-4", enabled to attempt linking at
all by 1ffff25) had no identity signal available -- transcript_too_short
skips GPT extraction entirely, so call_units is always empty for this
population. It fell back to "most recently updated incident on this
talkgroup", which silently misattaches a short ack to the wrong incident
whenever two are live on the same busy dispatch channel at once.
metadata_watcher.py already captures the P25 source radio ID (srcaddr) on
every call independent of transcript content, and it already reaches the
call doc (models.py, mqtt_handler.py:245) -- it was just never read by the
correlator. Thread it through _build_context, check it against the
srcaddrs already seen on each TG-matched incident before falling back to
recency, and accumulate it on the incident (_update_incident/_create_incident)
so later calls from the same radio can match.
Verified: 422 pass, 0 fail (4 new tests).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The rules/indexes deploy step already existed here, gated on `command -v
firebase` over SSH on the deploy VM. It never found one (no node on the
VM), so it silently warned-and-skipped on every single deploy for weeks —
PR #124 even auto-closed #13/#51 as if this were fixed, when it wasn't.
New standalone deploy-firestore-rules job runs on the Gitea runner itself
(always has node), authenticated via a new FIREBASE_TOKEN secret (from
`firebase login:ci`) instead of anything pre-installed on the VM. It's
independent of the deploy job's health-check/rollback chain on purpose —
a rules deploy failure has nothing to roll back and must not trigger that
logic. notify-failure now distinguishes which job actually failed so the
Discord alert doesn't misreport "production is unchanged" when the app
deployed fine and only the rules push failed.
Needs FIREBASE_TOKEN added as a Gitea Actions secret before this actually
runs — it will fail loudly (by design) until then, which is the whole
point: a loud failure beats a silent skip.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
intelligence.py skips GPT extraction for transcripts <=5 words ("10-8", "show
me clear", a unit check-in) -- real cost/hallucination guard, kept as-is. But
upload.py's no-scenes correlation fallback (the path that thin-links a call
by talkgroup even with zero extracted content) excluded ANY skip_reason,
including transcript_too_short -- so this exact population, brief but real
follow-up and clearance traffic, never even attempted to attach to anything.
Found live: "Live, Ossining." sitting an orphan 6 seconds before a real
incident's founding call, on the same talkgroup.
Now only garbage_transcript (Whisper hallucination, no real content) stays
excluded; transcript_too_short reaches the same thin/fast-path fallback
already trusted for no-transcript calls, gated the same way -- same-talkgroup,
recently-active incident required before anything attaches. No GPT re-invoked,
no new cost.
Verified: 410 pass, 0 fail.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
node-26#9, same shape as the ADS-B commit. Adds POST /telemetry/ais
(same node-key auth, same org_id-stamped upsert-by-key pattern, this time
by mmsi into a new `vessels` collection) and its docInMyOrg() firestore
rule. Frontend gets useVessels() (mirrors useAircraft(), longer staleness
window since AIS position reports are minutes apart, not seconds) and an
opt-in "Vessels" map overlay.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Logan CusanoandClaude Opus 5.5
logan