40014a47a354aeec135f908b9ef24cbc167e1ede
158
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Logan CusanoandClaude Opus 5.5
|
40014a47a3 |
c2-core: Archive "Load more" skipped 150 of every 200 calls
/calls/search scans a 200-row window and returns 50, but the next cursor was always the last SCANNED row — so with an unfiltered list each page jumped past the 150 matches it had already read and not shown. Resume after the last RETURNED row when matches overflow the page; keep the last-scanned cursor only when the page holds every match (the sparse- filter case that cursor exists for). Same fix for /calls/eval-queue. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Logan CusanoandClaude Opus 5.5
|
6479174022 |
frontend: date range picker on Incidents and Archive; fix Archive paging
Incidents and Archive get a from/to date range (native date inputs, local-day bounds). Incidents filters in the Firestore query; Archive passes date_from/date_to to GET /calls/search, which applies them as a started_at range — both ride the existing org_id/started_at index. Also fixes /calls/search and /calls/eval-queue paging: the cursor went to Firestore as a raw ISO string against a timestamp field, which compares by type rather than time, so "Load more" re-read the first page. Cursor and range bounds are now parsed to datetimes (400 on garbage). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Logan CusanoandClaude Opus 5.5
|
fa194e0f0a |
frontend: search, filters and load-more on Incidents; open Archive to viewers
Incidents page: text search (title, location, summary, units, vehicles, tags, location mentions), status and type filters, and a Load more button that pages the Firestore query 100 at a time. Filtering runs over the loaded window, and the page says so when older incidents exist. Archive (/calls): readable by every org member, not just admins. GET /calls/search now takes any Firebase token scoped to the caller's org — the Firestore rules already let members read every call in their org, so this widens nothing. Attach/detach stays admin-only (UI and routes). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Logan CusanoandClaude Sonnet 5
|
5f85a878fa |
admin: STT eval harness — record human-verified transcripts, measure real WER (#163)
Backend: three new routes on the calls router, deliberately separate from
PATCH /{call_id}/transcript (a production correction with real side effects
-- re-extraction, incident unlinking, vocabulary learning). This is pure
measurement and must never share that code path.
GET /calls/eval-queue -- calls with a transcript but no
eval_transcript yet, paged (same bounded-
window-plus-cursor shape as /search)
PUT /{call_id}/eval-transcript -- records eval_transcript/_by/_at only;
never touches transcript/transcript_corrected
GET /calls/eval-stats -- eval_count + average word error rate of
the raw and corrected machine transcripts
against the human-verified ones
internal/wer.py: standard word-level Levenshtein WER. Returns None (not 0.0)
when the reference is empty -- a call nobody transcribed must not score as a
perfect match.
Frontend: a new "STT Eval" tab on /admin -- one call at a time, audio player,
a textarea pre-filled with the machine transcript to correct into ground
truth, Save & next / Skip, running WER stats at the top. Built for working a
handful of calls at a time over however many sittings it takes, not a
one-shot form: the queue auto-refills from where the last save left off.
Verified: 438 pass, 0 fail (12 new backend tests). Frontend is UNVERIFIED --
this box has no Node.js/npm (confirmed absent), so neither typecheck nor the
dev server could be run. Matches existing code patterns and the CallRecord/
c2api types by manual review only.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Logan CusanoandClaude Sonnet 5
|
241a15b8da |
transcript_correction: reject a correction that changes a ten-code (#162)
correct()'s prompt says "Do NOT expand ten-codes" and "NEVER add information", but nothing checked the model's output against its own rules -- raw["corrected"] was accepted verbatim past a non-empty/changed check, and the "changed" field it returns was logged for audit and never validated. Caught live: the same call's raw vs corrected transcript showed "10-7" rewritten to "10-13" in one place and "10-4" in another, plus a bare "7" expanded into "ShotSpotter" -- alongside genuinely good fixes (Holmes Street and 4th and Rowe -> Home Street and Forest Ave, from this system's own vocabulary). A wrong 10-13 standing in for a real 10-7 reads exactly as trustworthy as a correct transcript, which is worse than leaving the raw mishearing in place. _code_tokens() extracts every ten-code/signal-shaped token from the original and corrected text/segments; any change to that set discards the correction and falls back to raw. Checked independently for the joined text and for segments, consistent with the existing all-or-nothing segment-alignment rule. Does not catch a wrong word swapped for another equally plausible non-code word -- that class still depends entirely on the model following its own prompt. Filed server-26#161 for the broader STT/audio quality initiative this belongs alongside. Verified: 426 pass, 0 fail (4 new tests). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Logan CusanoandClaude Sonnet 5
|
66bbf5b473 |
intelligence: don't reject a geocode just because it is far from the node (#159)
_geocode_location's node-distance fallback (geocode_max_km, 40km) is a
proxy for "is this plausible" that only makes sense when the node's own
position is the best guess we have at the area. It doesn't hold for a
citywide/patched feed: node-002 sits in Westchester but relays "New York
City - NYPD Citywide 2 Patch", ~56km from the addresses on it. Every real,
correctly-geocoded address on that talkgroup was rejected by this check,
every time -- location_coords stayed permanently null for the whole
system, which killed the location_proximity correlation signal and let
duplicate incidents form for the same event reported at two nearby
addresses two minutes apart ("Shots Fired at Jackson Avenue" /
"Shots Fired at 1108 Jackson Avenue", 2026-09-20 ~23:00 UTC, merged by
hand via the Archive page's attach/detach while this fix went in).
trust_named_region skips the node-distance rejection exactly when the
query already carries a place name that isn't the node's own position --
operator-set area_context, or a municipality parsed from the talkgroup's
own name. The anchor path is untouched; an anchor's own radius is always
authoritative when one has been resolved.
Also fixes a compounding defect found while verifying: the query was
grafting the node's own COUNTY onto an already-self-named region
("...New York City..., Westchester, New York"), which is self-contradictory
and could degrade the geocode independent of the distance check. The
node's county is now used only when nothing else names the place; state
stays in both branches since it's coarse enough to be correct either way.
Extracted the query-assembly logic into a pure, unit-tested helper
(_location_query_parts) rather than testing it only through the full
extraction pipeline.
Filed #160 as a follow-up: place_verifier.py's verify() has the identical
no-anchor gap for transcript place-name correction, not fixed here.
Verified: 422 pass, 0 fail (9 new tests).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Logan CusanoandClaude Sonnet 5
|
6c095083fc |
correlator: use srcaddr for thin-call disambiguation instead of pure recency (#158)
The fast/thin path (short acks like "10-4", enabled to attempt linking at
all by
|
||
|
Logan CusanoandClaude Sonnet 5
|
1ffff25cd2 |
upload: let short transcripts (<=5 words) attempt correlation instead of never linking at all
intelligence.py skips GPT extraction for transcripts <=5 words ("10-8", "show
me clear", a unit check-in) -- real cost/hallucination guard, kept as-is. But
upload.py's no-scenes correlation fallback (the path that thin-links a call
by talkgroup even with zero extracted content) excluded ANY skip_reason,
including transcript_too_short -- so this exact population, brief but real
follow-up and clearance traffic, never even attempted to attach to anything.
Found live: "Live, Ossining." sitting an orphan 6 seconds before a real
incident's founding call, on the same talkgroup.
Now only garbage_transcript (Whisper hallucination, no real content) stays
excluded; transcript_too_short reaches the same thin/fast-path fallback
already trusted for no-transcript calls, gated the same way -- same-talkgroup,
recently-active incident required before anything attaches. No GPT re-invoked,
no new cost.
Verified: 410 pass, 0 fail.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Logan CusanoandClaude Sonnet 5
|
3d2b722c64 |
Wire AIS end to end: telemetry ingestion + live map overlay
node-26#9, same shape as the ADS-B commit. Adds POST /telemetry/ais (same node-key auth, same org_id-stamped upsert-by-key pattern, this time by mmsi into a new `vessels` collection) and its docInMyOrg() firestore rule. Frontend gets useVessels() (mirrors useAircraft(), longer staleness window since AIS position reports are minutes apart, not seconds) and an opt-in "Vessels" map overlay. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Logan CusanoandClaude Sonnet 5
|
5537b095df |
Wire ADS-B end to end: telemetry ingestion + live map overlay
node-26#9. Adds POST /telemetry/adsb (node-key authed via require_node_service_or_firebase_token) that upserts one Firestore doc per icao into a new `aircraft` collection, org_id stamped from the reporting node the same way upload.py defensively stamps `calls`. firestore.rules gets a matching docInMyOrg()-gated read rule. Frontend: useAircraft() mirrors useNodes()'s onSnapshot pattern, filtering docs older than 2 minutes client-side since nothing prunes a stale aircraft doc server-side yet. MapView gets an opt-in "Aircraft" overlay (unchecked by default, like the weather radar layer) rendering a rotated plane glyph per sighting. Unverified via typecheck — no Node.js/npm on this authoring box yet (see CLAUDE.md Testing reality). Server side is pytest-covered (test_telemetry.py). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Logan CusanoandClaude Sonnet 5
|
8892e824fc |
Add second-SDR fields: secondary_sdr_mode + sdr_count on NodeRecord
Server-side half of node-26#9. NodeRecord gains secondary_sdr_mode
(none|adsb|ais|op25_2) and sdr_count; checkin ingestion stores both,
and PATCH /nodes/{id} accepts and re-pushes secondary_sdr_mode the same
way hardware_preset/ppm_override already work, so it isn't wiped by a
system reassignment (see server-26#111 for the pre-existing bug that
pattern avoids repeating).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Logan CusanoandClaude Sonnet 5
|
3f69879437 |
intelligence: stop the police-channel default from typing content-free chatter (#138)
EXTRACTION_PROMPT's incident_type rule always returned "police" for anything on a police channel unless contradicted, so pure administrative chatter (post check-ins, roll call, bare acknowledgements) got a truthy incident_type and defeated the creation gate's "type" veto (_call_is_substanceless) ~82% of the time (measured server-26#138/CORRELATION_REVIEW_0914.md, confirmed against live 9-14 data: 9/9 type-veto examples checked all had severity "routine" — same population the severity rubric already correctly identifies as content-free, incident_type just wasn't using that signal). Rule now checks for actual event content FIRST, on every channel, before applying the channel-inference defaults; content-free traffic returns "unknown" (already normalizes to None) instead of a channel default, letting the existing gate correctly veto it. No correlator/gate code touched — CORRELATION_REVIEW_0914.md's own recommendation was to fix this in the prompt, not _call_is_substanceless, to avoid risking a real event getting gated out. Verified: 401 pass, 0 fail. Live effect to be confirmed against fresh traffic (AI features just re-enabled this session after being off since 9/14) — tracking in SESSION_STATE.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Logan CusanoandClaude Sonnet 5
|
fb0bb15c22 |
correlator/intelligence: close the incident-clearance gap (dispatch-to-10-8 lifecycle)
Two independent fixes, found by tracing why incidents never actually close
(only 2/281 incidents in 5 correlation debug windows ever got a non-empty
units_cleared; 183 resolved via the 90-min idle sweep instead of a real clear):
1. Pattern B (re-dispatch accept, no explicit 10-8): reassignment=True already
fired correctly and suppressed the unit from re-linking to its prior
incident, but nothing ever released the unit FROM that incident — it just
sat "active" until the idle sweep timed it out. _release_reassigned_units
now scans other active incidents for unit overlap on a reassignment and
clears the unit there, reusing the same units_active/units_cleared merge
(factored out as _apply_unit_clearance) that explicit 10-8 extraction uses.
2. Pattern A (self-clear) extraction was inconsistent for two reasons: no
per-system unit ID format awareness anywhere in the pipeline (formats vary
by department with zero shared convention), and the cleared_units prompt
rule only accepted a unit self-reporting, missing dispatch confirming a
unit's status back to them. Added system.unit_format_hint (owner-authored
free text, GET/PUT /systems/{id}/unit-format, no auto-induction yet) fed
into the extraction prompt, and broadened the cleared_units rule while
still requiring an identifiable unit ID (guards against bare "10-8"/"clear"
noise, including Whisper hallucination runs already caught upstream by
_is_garbage_transcript).
Verified: 401 pass, 0 fail (local Linux venv ~/venvs/drb-5c — see CLAUDE.md
testing-reality note).
server-26#pending — not yet filed, Gitea unreachable from this sandbox.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Logan CusanoandClaude Sonnet 5
|
8dd636af8f |
correlator: stop the re-correlation sweep racing an in-flight upload (#131)
server-26#131: same call_id ends up in TWO incidents' call_ids, byte-identical extracted data, ~2% of linked calls across 3 live dumps. Root cause: the sweep's orphan filter (incident_id/incident_ids/corr_path all absent) can't tell 'never processed' apart from 'real-time pipeline is still mid-flight' -- a call whose STT/scene-extraction/correlation chain (routers/upload.py _run_intelligence_pipeline) hasn't finished yet has none of those fields set, so the sweep picks it up and correlates it independently, sometimes onto a different incident than the real-time path lands on. Confirmed in review: _update_incident/_create_incident append call_id to an incident's call_ids unconditionally, with no cross-incident dedup guard -- preventing the second correlation attempt is the only lever available at this layer. Fix: _run_intelligence_pipeline marks intelligence_started_at on the call doc before any slow step; the sweep holds back any call whose marker is under 15 minutes old (raised from an initial 5 -- see below), regardless of how orphaned it otherwise looks. No marker at all (pre-#131 call doc, or the marker write itself failed) is not held back -- absence isn't evidence of an in-flight pipeline, and that's #131's own pre-existing population. Also covers the /calls/{id}/reprocess path, which calls the same _run_intelligence_pipeline. 15 min, not 5: neither the OpenAI Whisper client nor the Gemini call in llm_correlator.py sets a request timeout (filed server-26#153), so 5 min was a guess against an unbounded tail -- drb-correlation-review flagged this. Raising it is free on the recovery side: a call that finished processing (linked or genuinely orphaned) always has corr_path set and is already excluded by the sweep's other filter, so this constant only ever delays calls that are still actually running. DEFERRED.md row 52 (outside this repo, Version 5C root) updated to flag its ~6 min timing figure as stale. recorrelation_sweep.py had zero test coverage before this. New file covers the guard function's boundary (age < threshold vs exactly-at vs old vs missing vs unparseable) and one integration-shaped test proving a racing call never reaches correlate_call while a genuinely-orphaned call still does. Sandboxed pytest: 381 -> 387. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
f23026b9ab |
correlator: address drb-correlation-review notes on #139
- Document call_severity's routine-coercion asymmetry with incident_type (extraction-said-routine vs extraction-said-nothing look identical). - Test docstring no longer overclaims the ctx-linkage it doesn't cover; points to the tests that do (test_consensus_gate.py, test_incident_identity.py). - scene1 now uses a distinct severity so the test actually exercises both fields symmetrically; the 'no flat top-level clobber' claim is now asserted, not just commented. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
7717fcccdd |
correlator/admin: capture per-scene incident_type + severity (#139)
_call_is_substanceless's "type" veto reads ctx["incident_type"] at decision time, but that value was never persisted per-scene — only the last-scene-wins flat field, which #138's window-4 dump analysis couldn't tell apart from cross-scene contamination without re-guessing from a live dump. Adds incident_type/severity to _apply_and_log's per-scene write and to admin.py's _scene_summary allowlist (the debug-dump reader has its own field allowlist, separate from the write side — silently would not have surfaced otherwise). Sandboxed pytest: 380 -> 381. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
422e9a4dc8 |
correlator: fix two comments left describing the removed tactical path (#134)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
b1884852d5 |
correlator: remove is_dispatch and the tactical fit path entirely (#134)
Full removal, not a hardcoded flag: _is_dispatch_channel, _DISPATCH_TG_RE, the is_dispatch parameter, and _call_fits_incident's tactical branch are gone. One evaluation path for every channel. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
0473e6a583 |
correlator: always run the dispatch-strict fit test, not name-guessed (#134)
is_dispatch was computed from _is_dispatch_channel(talkgroup_name) and picked between two _call_fits_incident evaluation orders: dispatch (requires a positive signal, runs location-conflict/content-divergence vetoes on unit overlap) vs tactical (skips both vetoes, defaults to True on no signal at all within 20 min). Per #133's reasoning, a name not literally containing dispatch/patched/primary got the unvetoed, default-True path solely because of its label. Hardcoded is_dispatch=True at its one real call site; the tactical branch and its own tests stay in place, unreached. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
c50bfda8db |
correlator: drop the dispatch/tactical name-guess from the escape hatch, use one window (#115)
Owner correction from direct scanning experience: a talkgroup named tac/tactical only sees materially different traffic during a real incident, and that's rare -- the bulk of traffic on any monitored channel, including high-risk stops and pursuits, runs on the main channel regardless of what it's named. _is_dispatch_channel's string match on the talkgroup name is a naming-convention guess, not a detector of actual channel behavior; trusting it here meant a busy single-channel department not literally named 'dispatch' would silently get the more permissive 15-minute window and could reproduce #115's original bug (the gate never firing on the channels it targets). Always use tg_dispatch_thin_idle_minutes (5 min) in the escape hatch, regardless of talkgroup name. Does NOT touch incident_correlator.py's own fast/thin idle-window selection, which uses the same dichotomy for a different, decision-changing purpose -- bigger blast radius, left for its own review. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
0fe6d3b567 |
correlator: delete stale scenes on re-extraction instead of leaving them to rot (#96/#114)
Review of #132 found a blocker: PATCH /calls/{id}/transcript wipes tags/severity/location/units/embedding before re-extraction, but not the new scenes map, and doc_set(merge=True) can only add/overwrite nested map keys, never remove one. A call corrected from 3 scenes to 1 kept scenes.1/scenes.2 with pre-correction transcripts and incident_ids forever -- corrupting the exact per-scene tally #96 exists to make trustworthy, and able to re-feed stale text into #114's summarizer fix if a stale scene's incident_id still names a real incident. Fix: fstore.doc_update(...,{"scenes": fstore.DELETE_FIELD}) -- a real delete, not a merge over an empty map. Added fstore.DELETE_FIELD (re-exports the real firebase_admin sentinel) and stubbed it in the sandboxed test conftest, which didn't have it. Also softened an overclaiming docstring: the Firestore nested-merge behavior is verified against the doc_set wrapper's pass-through, not against live Firestore. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
fae84a45c3 |
correlator+summarizer: per-scene call-doc storage, fixes #96 and #114's real fix
Every scene of a multi-scene call correlates independently in upload.py's scene loop, but every scene's corr_debug was written flat onto the same shared call doc — scene 2's write silently clobbered scene 1's corr_path/corr_consensus/etc (#96), and summarizer.py read the whole call's raw transcript per linked call, mixing text from scenes the incident had nothing to do with, while ignoring transcript_corrected entirely (#114). Fix: thread a scene_index from both `for scene in scenes:` loops in upload.py down through _correlate_with_consensus -> incident_correlator.preview_correlation/correlate_call -> _build_context -> ctx["scene_index"]. incident_correlator._apply_and_log now writes, in the same Firestore call: - the existing flat corr_* fields, unchanged (last-scene-wins, the safe backward-compatible default for any reader that doesn't know about `scenes` yet) - a new nested `scenes.<scene_index>` entry with {transcript, incident_id, corr_debug}, via doc_set(..., merge=True). Firestore's DocumentReference.set(data, merge=True) recursively merges nested map fields by key (documented SDK behaviour, not assumed) — a write to scenes.1 merges alongside an existing scenes.0 instead of replacing the whole `scenes` map. scene_index defaults to 0 for every caller with no scene concept (the recorrelation sweep, the no-scenes-extracted orphan-check path), so a plain single-scene call still gets a one-entry `scenes` map equivalent to reading its flat fields today. admin.py's _call_summary exposes the new `scenes` list per call (each entry carrying the same corr_* field names as the flat fields, so the two shapes are interchangeable to the tally); the summary tally now iterates each call's scenes-if-present, else its own flat fields, so a 2-scene call with two different corr_path values counts as two data points instead of one blend. New `scene_decision_count` sits next to `linked_call_count` to make that distinction visible. summarizer.py's _scene_text_for_incident reads a linked call's `scenes` map to find the scene(s) whose corr_debug recorded a link into the specific incident being summarized, joining more than one if several scenes landed in the same incident. Falls back to transcript_corrected-or-transcript for a call doc with no `scenes` field (predates this change) — the one-liner half of #114, worth doing regardless since it stops raw-transcript summaries even for old-schema docs. Does not touch #80/#95/#102's existing ctx-threading fixes (embedding/severity/coords/LLM-prompt-transcript) — correct as-is, out of scope here. Tests: 14 new (test_per_scene_call_doc.py, test_summarizer_scene_transcript.py, additions to test_admin_debug_correlation.py) covering the merge shape, last-scene-wins flat-field backward compat, the admin tally's per-scene vs per-call counting (including old-schema fallback), and the summarizer's scene-specific text selection (including old-schema fallback). Full sandboxed suite: 364 -> 378 passed, all green. Fixes server-26#96, server-26#114 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
3ae0bb2d5b |
intelligence: run the chatter classifier before the too-short skip, not after (#127)
82% of the classifier's backtest flags were <=5-word transcripts that already exit at skip_reason=transcript_too_short before the classifier ever ran, so shadow mode was on track to observe roughly a fifth of the real catch rate. Compute the verdict once, ahead of that check, and fold it into whichever doc_set already runs (no extra Firestore write). Also add a chatter_classifier_flagged/reason tally spanning both linked calls AND orphans in admin.py's summary block -- the target population is non-events, which land as orphans or single-call incidents, so linked alone undercounts it the same way corr_gate_veto would have without the #126 fix. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
05ddec8284 |
intelligence: shadow-mode upstream dispatch-vs-chatter classifier (server-26#127)
Three live measurement windows and two consensus-layer fixes (#125, #126) converged on one decision (CORRELATION_REVIEW_0907b.md, _0912.md): stop iterating the correlator's consensus layer, the actual lever is upstream — a classifier in scene extraction that recognizes radio housekeeping (roll call, bare 10-4/10-8/98 acknowledgements, unit check-ins) before it ever becomes a scene for the correlator to judge. Adds app/internal/chatter_classifier.py: a pure classify_chatter(transcript) function recognizing two shapes drawn from hand-labeled examples in the review docs, cross-referenced against the real dumps — not invented regexes. Deliberately conservative: anything that doesn't cleanly reduce to a known shape returns (False, None) and the existing pipeline runs unchanged. SHADOW MODE ONLY. intelligence.extract_scenes computes the verdict next to the existing _is_garbage_transcript / transcript_too_short gates and writes chatter_classifier_verdict / chatter_classifier_reason onto the call doc, but does not skip extraction. admin.py's correlation-debug _call_summary surfaces both fields, same pattern as corr_gate_veto (#115/#126), so the next live window can measure the real-world false-positive rate before anything is wired to actually skip extraction. TODO(server-26#127) marks the call site. Backtest against all three existing dumps (1002 calls): 154 flagged, 0 false positives (no flagged call carries tags, coords, non-routine severity, or matches any review-doc-named dangerous-to-drop transcript — the major extinguishing-fire call, geocoded calls, pursuit updates, the Pelham Station subject check, the property-retrieval call, all individually verified). tests/test_chatter_classifier.py: real transcripts from the dumps/review docs in both directions. Sandboxed pytest 332 -> 364, green. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
83beb2bf35 |
admin: surface corr_gate_veto on the correlation-debug endpoint (#115)
corr_gate_veto was written to corr_debug but the admin endpoint's whitelist (_call_summary + the summary tally) never surfaced it, so the last commit's whole point -- measuring window #4 instead of guessing -- would have produced nothing to read. Add it to both. Also softened the docstring's remaining overclaim: whether the active-only ctx[recent] limitation explains the 2/24 window-3 misses is unanswered, not confirmed -- read corr_gate_veto next window instead of asserting a guess again. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
598054746a |
correlator: mirror the dispatch/tactical idle split in the escape hatch, record the gate-veto reason (#115)
Review of #126 found: (1) the escape hatch applied tg_dispatch_thin_idle_minutes (5 min) unconditionally, but incident_correlator's own fast/thin path only uses that on dispatch channels and 15 min on tactical ones via _is_dispatch_channel -- mirrored the same selection here, plus a config.py note flagging the second consumer. (2) the docstring claimed a 'confirmed explanation' for 2 window-3 gate misses that was actually wrong (self-contradictory in its own text); replaced the guess with corr_gate_veto, written into corr_debug on every llm=orphan/rules=new disagreement that escalates, so window #4 can see *why* each one escaped instead of reconstructing it from the raw dump. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
400b74b519 |
correlator: shrink the same-talkgroup escape hatch from 2h to a few minutes (#115)
_recent_incident_on_same_talkgroup previously treated ANY same-talkgroup incident within the 2-hour correlation_window_hours lookback as 'recent', which disabled the whole LLM-orphan consensus gate on busy dispatch channels: window #3 (CORRELATION_REVIEW_0912.md) measured 0/24 gate fires against the exact target shape (rules=new, llm=orphan, tiebreak=new), with 22/24 explained by a same-talkgroup incident existing somewhere in the prior 2h — nearly guaranteed on channels producing 3-13 incidents/2h. Now the escape hatch only counts an incident as recent within settings.tg_dispatch_thin_idle_minutes (5 min), reusing the same recency bound the fast/thin path already uses for the 'dispatch, thin ack 10-30s later' case this hatch exists for, instead of inventing a new constant. Investigated the 2 unexplained misses (no same-tg incident found even by a naive full-collection timestamp scan): confirmed ctx["recent"] is built from status=="active" incidents with over-capacity incidents dropped (_build_context / _drop_capped), not a full collection scan — an incident that has auto-resolved or hit incident_max_calls/incident_max_duration within the window is invisible to this check even though it is chronologically recent. This does not explain the 2 misses (a same-tg incident was absent by both checks there, so some other _call_is_substanceless condition must be responsible), but it is a real gap in the check as written. Documented in the docstring with a TODO(server-26#115); fixing it needs a new, non-active-filtered Firestore query, out of scope for this pass. Tests: added a regression test proving an incident inside the old 2h window but outside the new 5-minute window now correctly gates (fails on main, passes here), plus a test proving a truly recent (<5min) same-tg incident still escapes the gate as intended. Sandboxed pytest: 327 -> 329 passed (2 new tests), all green. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
15a9d10666 |
correlator: keep the tiebreak for typed / reassignment calls in the orphan gate (#115)
_call_is_substanceless mirrored has_event_substance but not the creation gate's type-resolved short-circuit, so a routine-severity fire/medical call with no coords/tags/vehicles — or a reassignment (unit pulled to a new job) — could be gated to orphan where rules would open an incident. Bail out of the gate on incident_type or reassignment. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
dd426572fc |
correlator: fix consensus orphan-gate to test call substance, not empty corr_debug (#115)
The gate added in
|
||
|
Logan CusanoandClaude Sonnet 5
|
ca1d8fbdae |
correlator: gate LLM-orphan against rules-new instead of escalating to tiebreak (#115)
CORRELATION_REVIEW_0907b.md measured that radio housekeeping (unit check-ins, roll call, 10-8/10-98 clearings) is being promoted to incidents. Every case reads corr_llm_action=orphan, corr_rules_action=new, corr_consensus=tiebreak -> new: the cheap LLM correctly reads "not an incident", the rules engine says `new` only because there is no incident to link to, and the smart tiebreaker then sides with rules ~21/21. Reframing the tiebreaker prompt (#116) did nothing. The fix is a consensus-logic gate, not another prompt. Fix 1 (routers/upload.py) - LLM-orphan gate in _correlate_with_consensus: when the cheap LLM says `orphan` and the rules engine says `new` with NO positive event signal, resolve to `orphan` and skip the tiebreak call entirely. "No positive signal" = the rules corr_debug carries neither a positive corr_path (unit-continuity / location / fast/disambig / fast/single) nor a positive corr_fit_signal (unit_overlap / location_proximity). When it does carry one, the existing escalation-to-tiebreak is kept so a genuine event the LLM misreads as orphan still gets the second look. The resolved outcome records corr_consensus="llm_orphan_gate" (greppable, distinct from "tiebreak") and keeps corr_llm_reasoning / corr_rules_action / corr_llm_action populated. Fix 2 (incident_correlator.py) - tighten corr_path=location: the location path linked on a bare sub-location_proximity_km (0.5 km) distance with no unit or content check, which stitched a vehicle lockout to a station-restroom slip and merged two different churches an hour apart. A location link now requires unit overlap with the candidate OR a distance under a tighter bar (_LOCATION_TIGHT_PROXIMITY_KM = 0.2 km). Pursuit incidents keep their movement-speed-validated wide radius. A surviving location link now also writes corr_fit_signal (unit_overlap | location_proximity), consistent with Fix 1's positive-signal set. Tests: new tests/test_consensus_gate.py (13 cases) - the gate resolves to orphan without calling tiebreak on a no-signal disagreement; a unit_overlap / location_proximity / unit-continuity / fast-disambig rules signal still escalates; llm=link vs rules=new still escalates; the location path drops a shared-area candidate with neither unit overlap nor tight proximity, links on unit overlap, and links on tight proximity alone. Full c2-core suite 309 -> 322 passing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
d60fef67ad |
c2-core: add CORS middleware so the browser can call the REST API (#110)
The Archive page's GET /calls/search failed its CORS preflight (OPTIONS -> 405, no Access-Control-* headers). Allow the app origin(s) explicitly for the standard methods and the authorization/content-type headers. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix |
||
|
Logan CusanoandClaude Sonnet 5
|
1a631d65d0 |
correlator: address #116 review — call talkgroup id in the prompt, sort candidates
drb-correlation-review: ship, with two bounds the low-bar link rule needs. 1. _call_block emitted only the talkgroup NAME while _inc_summary emits numeric tg ids, so the "same talkgroup" precondition in _RULES was unevaluable and the low link bar applied unconditionally. _call_block now prints "Talkgroup: <name> (id <n>)". 2. ctx["recent"] is an unordered Firestore slice with no order_by; a busy 2h window (~40 active incidents) showed the model an arbitrary half of the candidates. _prompt_incidents() sorts by updated_at desc before the [:20] cap — also makes each row's idle: field monotonic. +2 tests. Full c2-core suite green (sandboxed venv). Review follow-ups (not blockers): _parse_response demotes an unresolvable link to orphan (drops the call) rather than falling back to rules — now on rising link volume; the 45% tiebreak escalation rate / smart-model cost is untouched; _ROAD_RE swallows leading tokens so "10 Parker Street" still won't road-overlap "Parker St". Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Logan CusanoandClaude Sonnet 5
|
3a944f35c1 |
correlator: give the LLM tier what it needs to link, stop it defaulting to "new" (server-26#115)
The 2026-09-07 measurement window (CORRELATION_REVIEW_0907.md) showed the consensus tiebreaker was the dominant over-split driver: it ran on 45% of calls and resolved link/orphan disagreements as "new" ~24/25 of the time, shattering one Mohegan Park car-alarm job into 9 incidents and opening ~7 incidents from radio checks / roll calls. Two causes, two fixes: 1. `_inc_summary` gave the model `id|type|loc|units|tags|idle` — no title, no talkgroup. It literally could not see that two "car alarms, Mohegan Park Ave/Avenue" incidents on TG 9560 were the same. Now includes the incident title (the strongest same-event signal) and talkgroup. 2. `_RULES` told the model "orphan when in doubt — conservative is always correct". For a system that over-splits, that is backwards: a wrong link is cheap, a duplicate incident is the failure. Rewritten to: prefer link for a plausible same-talkgroup continuation (low bar), reserve "new" for a genuinely different event, and explicitly "orphan" non-incidents (radio checks, roll call, 10-8/10-98, mileage logs). Plus `_extract_road_ids` now canonicalises street-type synonyms (Avenue→ave, Street→st, Road→rd, ...), so "Mohegan Park Avenue" and "Mohegan Park Ave" share a road id — that one difference was splitting the car-alarm incident. +tests/test_correlator_115.py. Full c2-core suite green (sandboxed venv). Bigger levers deferred to follow-ups: the consensus escalation itself (should a cheap-LLM "orphan" ever reach a tiebreak?), a first-class road-overlap fit signal in _call_fits_incident, geocode coverage. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Logan CusanoandClaude Sonnet 5
|
7189ba03e4 |
correlator: address #102 review — 0-based segment labels, never-empty slice
drb-correlation-review on the prior commit flagged two ways the per-scene transcript could silently fall back to the whole-call text: 1. _build_transcript_block numbered transmissions "1." while the prompt says "0-based indices" — a model echoing the labels it saw returned 1-based indices, shifting every scene's slice by one. Labels are now "0." to match the documented contract (also fixes the same latent skew in _build_scene_embed_text / #80). 2. An empty join (bad / out-of-range / non-int indices) hit `transcript or call_doc.get(...)` in _build_context and fell back to the whole-call transcript — re-opening the leak exactly when indices are wrong. The slice now falls back to this call's own whole transcript *before* _build_context sees it, so it is never "". Non-int and negative indices are rejected rather than raising. Slice logic extracted to `_scene_transcript_text` with a dedicated test file (4 cases: subset, corrected-wins, no-indices fallback, bad-indices fallback). Call-doc fallback kept (sweep / no-scene path) per the review. Also restored the `-> ` spacing lost in the prior commit's kwarg edit. Full c2-core suite green: 300 passed (sandboxed venv). Still DO NOT MERGE until the measurement window closes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Logan CusanoandClaude Sonnet 5
|
ef1e3d7f9d |
correlator: LLM tier reads the scene's transcript, not the whole call (server-26#102)
The last leg of the #80/#95 scene-context leak. llm_correlator._call_block read call_doc's whole-call transcript for every scene, so on a multi-scene call every scene's cheap-tier and tiebreaker decision was made against text that also contained the other scenes. - intelligence.py: each processed[] scene now carries its own "transcript" — transcript_corrected, else this scene's segments joined, else (single scene) the whole transcript. - _build_context / preview_correlation / correlate_call: take a `transcript` param; _build_context resolves ctx["scene_transcript"] from it, falling back to the call doc (sweep, single-scene, tests) — the fallback is kept here, unlike embedding/severity, because a scene always has real text. - upload.py: both scene loops pass scene["transcript"]. - llm_correlator._call_block: reads ctx["scene_transcript"] (call-doc fallback retained for test-built ctx). - recorrelation_sweep: passes the call doc's text explicitly. - +1 regression test. Full c2-core suite green (296 passed, sandboxed venv). NOT for merge until the running correlation measurement window closes and its dump is analysed — deploying a correlator change mid-window would mix old and new behaviour in the sample. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Logan CusanoandClaude Sonnet 5
|
a9d1d2475a |
correlator: judge each scene on its own embedding + severity (server-26#80, #95)
intelligence.py writes only the primary scene's embedding and severity to
calls/{id}. _build_context read them back off the call doc, so every
non-primary scene of a multi-scene call was correlated against scene 1's
semantic vector and severity rung: a scene about a different event scored
on the embedding path against the wrong incident, and could inherit a
minor/moderate/major severity it never had, clearing the creation gate on
borrowed weight. Same defect and same fix as the #87 coords leak.
- _build_context / preview_correlation / correlate_call: take embedding and
severity as params; drop the call_doc.get() fallbacks. A scene that
passes none has none, and is judged thin on its own signal.
- upload.py: both scene loops pass scene["embedding"] / scene["severity"];
_correlate_with_consensus forwards them. The no-scene unclassified branch
passes neither (correct: no scene, judged thin).
- recorrelation_sweep: passes the call doc's stored values explicitly
(whole-call re-link, link-only, so a borrowed severity cannot create).
- intelligence.py: SCENE DETECTION prompt tightened toward one scene
(server-26#5, partial) - MULTIPLE only for genuinely separate events,
"when unsure, one scene", plus a not-a-new-scene list.
- test_incident_identity.py: +2 regression tests mirroring the #87 test.
Full c2-core suite green (295 passed). #5 prompt change is unmeasured -
needs a scoped correlation-only window. Known remaining legs, tracked
separately: llm_correlator._call_block still reads the whole-call
transcript per scene; content-divergence veto skips on a None embedding.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Logan CusanoandClaude Opus 5
|
8b6c170265 |
Close viewer-triggerable OpenAI spend on incident summarize (server-26#81)
POST /incidents/{id}/summarize was gated by require_service_or_firebase_token,
which accepts any authenticated Firebase user including role "viewer". That
route spends OpenAI credits via the background summarizer. The call-side
equivalent was already moved to require_admin_token; this brings the incident
side in line with it.
The frontend's two "summarize now" buttons on the incident detail page are
already gated behind isAdmin, so this backend change matches existing UI
behavior exactly and does not break any viewer/operator surface — it only
closes direct-API access for non-admins.
Swept every other route in incidents.py: list/get are reads with no spend and
correctly stay open to any signed-in user; create/update/delete/link/unlink
were already require_admin_token. No other sibling route needed changing.
Adds test_incident_summarize_auth.py pinning the dependency wiring directly
(the convention used in test_admin_feature_flags.py), so a future revert back
to the weak dependency fails a test immediately.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Logan Cusano
|
bdb57ae75a |
correlator: stop non-primary scenes inheriting the call doc's pin (#87)
_build_context fell back to call_doc.get("location_coords") whenever a
scene passed no coordinates of its own. One radio call can be split
into several scenes, but only the primary scene's geocode is ever
written to the call doc — so every non-primary scene silently
inherited the primary scene's pin. That fabricated location_proximity,
the strongest accept signal the correlator has, for a scene that had
no location at all, and drove it into the primary scene's incident on
a pin it never had.
Drop the fallback: coords = location_coords. A scene with no location
is now correctly judged thin, cannot win the location path, cannot
supply call_coords to _call_fits_incident, and cannot seed
_find_cross_system_parent.
recorrelation_sweep.py, the only other caller of correlate_call, was
verified to already pass both location and location_coords explicitly
from the call doc, so the fallback there was a no-op and this change
is behavior-preserving for that path.
Adds test_a_scene_with_no_location_does_not_inherit_the_call_docs_pin
to test_incident_identity.py, pinning ctx["coords"] is None and
ctx["is_thin_call"] is True when location=None but the call doc
carries a location_coords.
Ref: server-26#87
|
||
|
Logan CusanoandClaude Opus 5
|
865b5b4317 |
Close the /admin/features side-door that needed a container shell to flip AI spend
Board minutes #62 Decision 2 (server-26#64), due 2026-08-31. CTO draft #60 finding 1 and CISO draft #61 finding 3 reached this independently. GET/PUT /admin/features accepted only a Firebase admin token, so the unattended runbook had no headless path and SSHed into the c2-core container to write config/ai_features with the admin SDK. Moving a platform-wide AI cost switch required a full container shell, and set_flags() wrote no audit entry either way, so a flag flip was unattributable however it happened. - New agent_service_key (AGENT_SERVICE_KEY), deliberately separate from the Discord bot's service_key. Sharing one key would collapse two principals into a single unattributable identity in every log line, and the bot has no business flipping AI flags regardless. - require_agent_key_or_admin accepts the agent key or a Firebase admin, and rejects the Discord key. The "key is configured" guard is load-bearing: compare_digest("", "") is a match, so a deployment that never set the key would otherwise accept an empty credential. - set_flags() writes an audit_log entry with before/after values and the actor, wrapped so an audit failure cannot lose the flag write or 500 the route. - Cascade helper sets the global doc and every system carrying an ai_flags override in one call. A global False already beats everything, but a system False beats a global True, so turning AI *on* could half-apply and leave a radio system hot after shutoff. It scans for the override rather than hardcoding the two known system IDs, so a new system cannot silently defeat it. - cascade defaults to False. PUT /systems/{id}/ai-flags and the AiFlagsPanel toggle mean a per-system override is deliberate operator intent; cascading by default would erase it on any unrelated global flip. The runbook opts in. Issue items 5 and 6 (retiring the SSH path from drb-worksession.md) are NOT done here and the runbook is untouched. The credential does not exist in production yet, so the SSH path is still the only one that works; retiring it now would break the next unattended run. Owner activation is recorded on #64. Tests 273 -> 289. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Logan Cusano
|
0635de8dac |
Stop alert webhooks putting raw transcripts in a third-party channel
Alert dispatch attached a 200-character raw transcript snippet to the alert_events document and POSTed the same text to the org's Discord webhook, with no redaction of any kind. Board minutes #42 ratified that person names are suppressed on every surface until E&O is bound, and a Discord channel is the least recoverable surface there is: once the text lands we do not own it, cannot unsend it, and cannot audit who read it. Raw transcript text now requires two independent gates, both closed by default: 1. alert_transcript_snippet_enabled -- an operator switch in config, set from the environment. 2. alert_snippet_opt_in on the org document -- the customer's own explicit consent. Gate 1 is not redundant. The frontend reads and writes Firestore directly from the browser, so the org flag alone would let an org owner opt themselves into receiving person names lifted from live public-safety traffic. Capability is the operator's to grant; consent is the org's. The gate fails closed on a Firestore error and on a call with no org (a pre-tenancy node that has not been backfilled) -- a less informative alert is cheap, an unrecallable disclosure is not. Alerting itself is unchanged: the webhook still fires and still names the rule, the talkgroup and the matched keywords. This does not wait on the Gate B3 redactor (#43, 2026-09-30). The snippet was a convenience field and needed no redactor to withhold. Tests assert the person name in a sample transcript does not appear in either the outbound payload or the Firestore write, in every combination of the two gates. Closes server-26#85. Refs #42, #43, #48. |
||
|
Logan CusanoandClaude Opus 5
|
e30d594eea |
Stop a back-dated call from silently disabling every recency gate (server-26#74)
_call_fits_incident measured incident idle with the signed helper while every other recency gate in the file uses the unsigned one. On the re-correlation sweep, `now` is the call's own started_at, which can precede the incident's last activity, so the value went negative. Negative idle made `idle_min >= 15` false, which meant the content-divergence veto never ran and unit overlap was accepted unconditionally -- on a shared dispatch backbone that is the feedback loop that lets one incident absorb a whole talkgroup. It also made `idle_min < 20.0` true at any back-dating, so a tactical channel returned tactical_default for every swept orphan out to the 90-minute bound. One variable feeds all four gates in the function, so this is a one-line change at the source. The signed value is untouched where it belongs: callers still compute corr_incident_idle_min themselves, so debug output keeps its meaning. Direction is toward more splitting, on the sweep path only, which is the point -- the bug was suppressing an over-merge veto. Forward-dated calls and anything inside the thresholds behave exactly as before. Two tests added alongside the existing idle-gate cases; both fail on the old line and pass on the new one. 266 pass, 0 fail. Refs server-26#74, #5, #80. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Logan Cusano
|
d18e4f0743 |
Make "AI is off" true, and stop the transcript PATCH from destroying calls
config/ai_features was not the switch it was documented to be. Three paths spent money with it off, and one path read it wrong, so per-system opt-outs did not opt anything out. - Correlation in the ingest pipeline tested the raw global flag instead of the per-system resolution. With a system opted out, extraction was skipped but the no-scenes fallback still correlated the call with empty tags, taking the thin/recency path and attaching it to whatever incident was most recent on that system. The opt-out did not disable correlation, it disabled good correlation and left the worst kind running. (#75) - Transcript correction ran on every transcribed call gated only by an env var, spending Gemini tokens and a Places lookup per proposed location. An "STT-only" window was never STT-only and its cost could not be attributed. Now behind transcript_correction_enabled. (#76) - _run_extraction_pipeline and the vocabulary learner, both reachable from PATCH /calls/{id}/transcript, checked no flags at all. (#76, #81) The flag resolver now lives in feature_flags.resolve_flags() rather than as a local helper in upload.py. Three copies of that logic is how #75 happened. PATCH /calls/{id}/transcript now refuses with 409 when correlation is off. That route wipes tags, severity, location, units, embedding and unlinks the call from every incident before queueing re-extraction. Gating extraction alone would have made it destructive-only in the standing flags-off configuration: the call left blank and orphaned forever, with the route still answering 200. The wipe and the rebuild are one transaction in intent, so it refuses before the first write. Also: the summarizer's stale-incident sweep is no longer behind summaries_enabled. It is pure Firestore with no model call in it, and gating it meant nothing auto-resolved while AI was off - so every incident stayed active forever and the candidate set every correlation reads kept growing. transcript_correction_enabled is documented as NOT a pure cost lever. The corrector is also the noise gate that sets not_speech; with it off, recogniser noise reaches extraction as a real transcript, comes back thin, and auto-attaches. Never open an evaluation window with correction off and correlation on. 14 tests added covering flag precedence, both pipeline paths, the 409, the correction gate and the summarizer no-op. Suite: 264 passed. Refs #75, #76, #81, #45. |
||
|
Logan CusanoandClaude Opus 5
|
cc038e6326 |
A unit call-sign is not a place
"Post 1-2" reached the geocoder, resolved against its talkgroup anchor and produced a confident pin in the right town for an event with no known location — while sitting in the same incident's `units` list the whole time. A plausible wrong pin is worse than no pin: nothing downstream can tell it is wrong. Extraction returns `location` and `units` from one pass, so a string in both is a misclassification, not two facts. Drop it before the geocoder sees it. Closes server-26#52. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Logan CusanoandClaude Opus 5
|
964343c819 |
area_context v2 + Maps place verification (server-26#36, #37)
#36 — the correction pass shipped in
|
||
|
Logan CusanoandClaude Opus 5
|
58efdbd6eb |
Correct the transcript before anything reads it
Correction existed, but as a line in intelligence.py's EXTRACTION_PROMPT --
which put it in the wrong place twice over. The same model call that extracted
units, location and severity emitted the correction afterwards, so extraction
reasoned over text already known to be wrong; and it sat behind
correlation_enabled, so during a cost-controlled STT-only window nothing was
ever corrected at all. That is the normal state during development.
internal/transcript_correction.py is now its own pass, between the degenerate
filter and the Firestore write. It receives an already-produced transcript plus
a reference list, so unlike a Whisper prompt it has no series to extend -- the
distinction that keeps vocabulary out of the recogniser's prompt, where an
enumerated ten-code list once made it hallucinate ten-code runs.
Reference data is merged from the talkgroup and the system, TALKGROUP FIRST. A
system spanning several counties can have a talkgroup covering one
municipality, and that municipality's streets must not be buried under a
county-wide list. A single-municipality system is the degenerate case: populate
the system level and every talkgroup inherits it. Area context is now SET --
municipality, county, roads, landmarks, on both scopes -- rather than guessed
from talkgroup names, which is what vocabulary_learner did and which is close
to useless across multiple counties.
Segments are corrected too, not just the joined text. extract_scenes builds its
prompt from numbered segments whenever there is more than one, so a correction
that only fixed the transcript would have been discarded on exactly the
multi-transmission calls carrying the most content. Alignment is enforced: an
array of the wrong length or type is dropped whole, because scenes map back to
transmissions by index and a shifted array would misattribute audio silently.
Whisper is also retried once on degenerate output. Call e49ea32c produced a
56-word ten-code counting run on one attempt and ordinary speech on the next --
same clip, same temperature=0 -- so a hallucination is a coin-flip, and
discarding on the first bad roll threw away a recoverable transcript.
Two things found on the way:
PUT /systems/{id} wiped ten_codes on every save. The systems form sends only
{name, type, config}, and model_dump() wrote every omitted field as its default
over the top. Now exclude_unset. area_context would have been the next victim,
which is why it gets its own route alongside ten-codes rather than a field on
that payload.
Closes server-26#36.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Logan CusanoandClaude Opus 5
|
1bfa856d1b |
Serve audio as whatever it actually is
audio/mpeg was hardcoded at both points call audio is written and served, from back when the node produced nothing but 16 kbps MP3. It now uploads FLAC, and a browser will not play a FLAC body labelled audio/mpeg. storage.py grows one extension -> Content-Type map, used by the GCS upload and by /media. Keyed off the object's real extension, so every existing .mp3 recording keeps working with no migration -- and _safe_audio_filename already accepted .flac, so object naming needed nothing. Also flags what this costs: /media sends the whole body with Accept-Ranges: none, which was fine at ~60 KB per call and is not fine at ~1.3 MB/min. Noted at the header and in DEFERRED.md, whose stated reason for deferring Range support was the old file size. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Logan CusanoandClaude Opus 5
|
140dfbfc74 |
Give the archive a real read, and the debug view a verdict
Three backend pieces the /calls page needs, plus the fix for a debug view that
hid its data exactly when it was wanted.
GET /calls/search — paged, filterable call archive. GET /calls returns every
call in one unordered shot: fine for a node's handful of active calls, useless
as an archive. Only the org scope and the started_at ordering go to Firestore,
since that pair is the one composite index that exists; the rest filters in
Python over a bounded window, the same shape admin.py's debug route uses. The
cursor advances over the scanned window rather than the returned page, or a
sparse filter would re-scan from the same place forever.
Manual attribution. POST /incidents/{id}/calls/{id} only ever wrote the legacy
scalar incident_id, never incident_ids -- which is what the correlator writes
and what the frontend queries with array-contains. A manually attached call was
therefore invisible on the incident page it had just been attached to. It now
maintains both and marks the summary stale. DELETE is new: there was no way to
undo an attachment at all, so a wrong link was permanent.
The debug view no longer filters to AI-enabled systems by default. That filter
emptied the view the moment the flags went off, which is precisely when a
window gets reviewed -- on 2026-08-23 it fell from 100 incidents to 6 between
switching correlation off and opening the tab. ai_systems_only=true restores it.
It also returns a summary block now: corr_path / fit_signal / consensus /
llm_action tallies, transcript coverage on both linked and orphaned calls,
single-call and median-calls-per-incident for fragmentation, max span and
anything past the server-26#22 caps for merging, and the count of incidents
still carrying a fallback "— TGID" title. All of it was being recomputed by
hand from the raw payload on every review.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Logan CusanoandClaude Opus 5
|
039a06dc72 |
Let C2 name a talkgroup it already knows
84 of the 100 incidents in the 2026-08-23 dump were titled "Ems — TGID 9048"
or "Other — TGID 9600" -- the fallback, not a description. The title is the
incident's name everywhere it appears: list rows, map pins, Discord alerts.
_create_incident builds it from a content tag and a talkgroup label, and the
label was collapsing to "TGID {id}" because talkgroup_name arrived as None.
It is a plain form field on /upload, forwarded untouched into correlation, and
the node only sends it when OP25 had the name in its loaded tags file -- which
is exactly the case C2 can cover from its own systems collection, where all 125
talkgroup definitions live.
The lookup already existed, on the other path: mqtt_handler resolved it from
the system config on call_start. So the call document held the right name while
the pipeline that titles the incident ignored it. That asymmetry is the bug.
internal/talkgroups.py is now the one implementation -- caller's hint, then the
call document, then the system config -- and both paths use it.
_run_intelligence_pipeline resolves once at the funnel /upload and
/calls/{id}/reprocess share, so the dispatch-channel test, scene extraction and
the title all see a real name. When the call document was the thing missing it,
the resolved name is written back, so the archive and the orphan panel stop
showing a bare TGID too.
Also gives fast/thin a corr_fit_signal. It is 63% of all links and was the only
path writing none, so corr_fit_signal was absent on 295 of 309 calls and the
admin debug view's distribution panel read empty -- looking broken when it was
faithfully reporting that the dominant path records nothing. It now says
thin_recency, which is what actually decided it.
Closes server-26#34. Refs server-26#35 -- the tier's 3.5% invocation rate is a
cost/benefit question, not a bug, and stays open.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Logan CusanoandClaude Opus 5
|
82c88379d4 |
Stop an incident lying about what it is and where it is
An incident header had two independently last-write-wins halves, and in the
2026-08-20 dump both were wrong at once. `b9b4f392` opened on a suspect search
at 80 Grasslands Road; it was labelled "100 South Mosher" (its third call),
pinned at `Westmed` (its second), and titled after the label. Five of six
incidents were pinned somewhere other than the place they claimed to be.
Location and pin are now one value
----------------------------------
`_resolve_location_pair()` computes `location`, `location_coords` and the new
`location_coords_source` together, and `_update_incident`/`_create_incident`/
`_create_master_incident` always write all three. There is no longer a code
path that can move one and leave another behind — including the cross-system
master, which used to take its label from the parent and its pin from the call.
The pin now carries the label it was geocoded from. `_verified_pin()` returns
it only when that source still matches the incident's current label; anything
else is dropped. That includes every pre-existing incident, whose pin has no
recorded source and therefore cannot be reconciled — which is the right
outcome, since the dump says 5 in 6 of those are wrong. A missing pin reads as
missing data; a wrong pin reads as fact, and this is a map people may act on.
An incident also keeps the first place it was given rather than the latest.
Later mentions still accumulate in `location_mentions` (what the map path is
drawn from); they just don't rename the incident's own location. The one
permitted change is filling in a pin the incident never had, from a later call
naming the exact same label — geocoding needs the node position, a quota and a
response, so the same address genuinely does fail once and resolve later.
"49" is not a place
-------------------
`clean_location()` rejects any string with no two-letter word in it, applied at
extraction (intelligence.py, before the geocoder and before the call document)
and again at the correlator's context boundary. `9d376ffe` carried
`location: "49"` from "Fire received. Flames from 49." — a box number — and its
summary asserted "A fire incident was reported at location 49". Nothing
validated that field at all, so it would have recurred.
Title: the founding event, escalation only
------------------------------------------
The title was re-derived from the newest classified call, so `f5190670` was
named after the thirteenth of its thirteen events. It now names the call that
opened the incident, recorded in `title_tag`/`title_severity`, and can only be
replaced by a call of strictly higher severity.
Three candidates were considered:
* Newest call (status quo) — rejected. The same incident has a different name
at different times, so a user who saw it in the rail cannot find it again,
and the name is decided by radio timing rather than by the event.
* Highest severity alone — rejected as the sole rule. Severity has four
levels and most traffic sits on one of them, so ties are the common case
and the tiebreak degrades to "newest" — the defect it was meant to fix.
* Founding event, escalated by strictly-greater severity — chosen. An
incident's identity is the event that opened it, so that is its default
name and it is stable for the incident's whole life. The single case where
the header MUST change is the one where the situation got worse: a check
condition that becomes a structure fire is a structure fire, and the
worst-first rail, the "Major only" filter and the map colour all exist so
that is never missed. Requiring strictly-greater makes it monotonic, the
same contract `_max_severity` already gives the severity field: routine
chatter can never take the name back.
A summary-level title regenerated as a whole was rejected outright: it needs an
LLM call per incident, AI flags are off in production, and every incident today
would have no title at all.
Two renames survive, because neither replaces an event name: filling in the
placeholder title of an incident that opened on a call with no content tags
("Police — Ch 1"), and re-rendering the same event once the incident learns its
address. Incidents created before this change have no `title_tag`, so their
existing title is treated as the founding one rather than handed to whichever
call links next.
Interaction with the caps from
|
||
|
Logan Cusano
|
c7be6416f2 |
Surface LLM correlation fields in debug view; fix unit-continuity path
/admin/debug/correlation stripped corr_consensus and the corr_llm_* fields
that upload.py's consensus correlator writes onto the call doc, making it
the one tool built to answer "is the LLM correlation tier alive" unable to
answer it (2026-08-19 dump had to infer LLM state from commit dates instead
of reading it off the data). admin.py's _call_summary() now includes
corr_consensus, corr_llm_reasoning, corr_llm_action, corr_rules_action.
The unit-continuity correlation path never wrote corr_matched_units, unlike
fast/single and fast/disambig, so the debug view showed null for a match
that was in fact unit-driven by construction. Now populated unconditionally
on that path (server-26#16).
Also traced the negative corr_incident_idle_min (-4.1 observed) to its root
cause: the re-correlation sweep anchors `now` to the linking call's own
started_at, and that back-dated value was being written straight into the
incident's updated_at, letting it land before the incident's own
started_at. Added _floor_at_started_at() so updated_at can never precede
started_at. (commit
|