Commit Graph
112 Commits
Author SHA1 Message Date
Logan CusanoandClaude Opus 5.5 d2437e3a0b Picker review fixes: validate stream_id on stop; accurate offline + no-system copy
- POST /discord/nodes/{id}/streams/stop: stream_id must be guild:channel
  snowflakes, so a malformed id is a 422 instead of a Firestore path error.
- Offline footer said channels resume on reconnect; ad-hoc ones stop once
  the node is gone past the lease grace (reconcile_once). Say so.
- Picker: say why Start is disabled when the node has no system.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 21:36:30 -04:00
Logan CusanoandClaude Opus 5.5 a2c840f186 Picker: share PR #218's guild_name field and its no-fallback-after-unlink rule
A server an org explicitly unlinked (unlinked_at) must not fall back to the
default org in the picker either, or the founding org could stream into it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 20:03:44 -04:00
Logan CusanoandClaude Opus 5.5 62e6d69bfa Node page Discord picker: org's linked servers + voice channels, not raw IDs (#213 item 3, #109 item 13)
- c2-core routers/discord_web.py:
  PUT  /discord/guilds/{id}/directory (service key): the server bot pushes each
       server's name + voice channels; merged onto discord_guilds, never touches org_id.
  GET  /discord/nodes/{id}/join-options: the node org's servers (same
       resolve rule as the bot), their voice channels with any live stream,
       radio-bot readiness + invite links, and this node's streams. No tokens.
  POST /discord/nodes/{id}/streams: stream pinned to the node; server must be
       the node org's, channel must be in the bot's directory, guild + node caps.
  POST /discord/nodes/{id}/streams/stop: one stream on this node, or all.
  Control = platform admin, org owner, or operator owning the node.
- nodes.py: legacy raw-ID discord_join now refuses a server outside the node's org.
- discord bot: DirectorySync cog pushes on ready (15-min loop), join/leave,
  rename, voice/category channel changes (debounced).
- frontend: NodeDiscordRadio panel + picker replaces DiscordJoinModal and the
  admin-only Join/Leave buttons.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 20:02:16 -04:00
Logan CusanoandClaude Opus 5.5 e3c68738e8 Merge origin/main into feat/discord-system (keep cameras + discord routers)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 18:32:26 -04:00
logan 8d9fe01828 Merge branch 'main' into fix/153-ai-request-timeouts 2026-10-04 18:29:00 -04:00
logan 1d7f4af42f Merge branch 'main' into feat/live-ops-wall 2026-10-04 18:28:36 -04:00
Logan CusanoandClaude Opus 5.5 a31a8df17c Discord radio channels: wire leases + desired-state streams (server-26#203, #204)
c2-core
- discord_streams/discord_leases wired in: POST /discord/streams,
  POST /discord/guilds/{id}/stop, GET /discord/guilds/{id}/overview,
  GET /discord/streams/{id}, PUT /discord/guilds/{id}/link (admin),
  GET /discord/node/{node}/tokens/{id} (node key, lease holder only).
- Retained nodes/{id}/discord/desired (token ids, never token strings),
  nodes/+/discord/actual subscribed and handled; per-node publish lock keeps
  the retained list the newest; failed publishes retried by the reconciler.
- Tokens freed only after the node acks logout (or offline past
  discord_lease_offline_release_seconds) - sweeper and C2 startup no longer
  release leases.
- /nodes/{id}/command discord_join/leave map onto streams; CommandPayload
  keeps preferred_token_id. /tokens: admin-only list, validate on add against
  Discord, refresh, flush via transactional force release.
- Guild -> org via discord_guilds, unlinked guilds fall back to
  DISCORD_UNLINKED_GUILD_ORG_ID (default "founding").
- tests/fake_firestore.py: optimistic-transaction fake; 27 lease/race tests.

discord bot
- /join, /leave, /status, autocomplete scoped to the calling guild; /leave
  never touches another server; Manage Channels default permission;
  /join edits its reply with the node's real result; /help copy fixed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 18:05:26 -04:00
Logan CusanoandClaude Opus 5.5 278edf48dc Explicit timeouts on every OpenAI and Gemini request (#153)
No AI client set a timeout: OpenAI's SDK waits 600s per attempt and retries
twice (timeouts included); google-genai waits forever. A hung provider call
had no upper bound, so recorrelation_sweep's 15-minute pipeline hold-back was
a guess against an unbounded tail.

- config: stt_timeout_seconds (120), openai_timeout_seconds (60),
  openai_max_retries (1), gemini_timeout_seconds (60).
- app/internal/ai_clients.py builds every OpenAI/AsyncOpenAI client with them
  (transcription, extraction, embeddings, summarizer, vocabulary learner,
  trip chat). gemini.py's shared client gets HttpOptions(timeout=ms).
- A timeout raises like any other provider error, so existing fallbacks are
  unchanged: rules-only correlation, rules decision on tiebreak, raw
  transcript on correction, ai_health "transient".
- MIN_MINUTES_SINCE_PIPELINE_START is now derived from those timeouts
  (pipeline_worst_case_seconds, 3-scene budget + 2 min non-AI slack), floor
  15: 21 min with the defaults. Loop warns if it reaches the scan window.
- tests/test_ai_timeouts.py: client limits, a real hung-socket request that
  gives up in <5s, timeout fallback per tier, the derived bound, and a guard
  against any new OpenAI( outside ai_clients.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 18:00:25 -04:00
Logan CusanoandClaude Opus 5.5 73147803ea Reject revokes the node's MQTT login and API key (server-26#195)
reject_node only set approval_status, so an approved-then-rejected node kept
its dynsec login, its /upload key and /systems read access.

- reject_node: status -> rejected first (fail-closed), then
  dynsec.delete_node_client, then delete node_keys. dynsec failure -> 502,
  key kept, retry is idempotent. Re-approve mints a fresh key.
- upload.py: valid key alone is not enough; node must be approved (403),
  uncached read so revocation bites immediately.
- dynsec.reconcile_all: deletes rejected nodes' logins on startup, healing
  nodes rejected before this change or whose revoke failed.
- Reject confirm text says it revokes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 17:58:51 -04:00
Logan CusanoandClaude Opus 5.5 aa5b71d853 Live ops wall + pluggable camera providers with licence gating (server-26#198)
Backend (c2-core):
- app/internal/cameras/: provider interface, licence model, per-provider
  catalog cache; one adapter per source in design-canvas/camera-sources.md.
  Implemented: 511NY (legacy shape + tolerant XML inventory parser, 511x
  endpoints UNVERIFIED), Caltrans, DelDOT, Ontario 511, DriveBC, Calgary,
  TfL, NSW, Fintraffic. Stubs: NYC DOT, Westchester DPW, TRANSCOM, CTroads,
  511NJ, Austin, TxDOT, MD CHART, WSDOT, Tallinn, Tarktee, Warendorf,
  TrafficLand, Windy, EarthCam.
- Every provider defaults OFF (config/camera_providers). Gated licences
  (agreement required / unverified / non-commercial / paid) need an explicit
  acknowledgement stored with who/when and written to audit_log; the audit
  write is a precondition. Stubs and prohibited sources cannot be enabled.
  Ack is bound to the licence status it was given for.
- GET /cameras (signed-in), GET/PUT /admin/camera-providers (admin only).
- /traffic/511 follows the ny511 provider switch and returns the DAA credit
  + NYSDOT as-is disclaimer; 511NY event descriptions no longer truncated
  ("may not alter the substance of a report").

Frontend:
- LiveView becomes the ops wall: Region > Incident > Camera drill-down,
  worst-first rail (MapView), KPIs, transcript ticker, scrubber incident
  bands, auto-tour, deep links (#inc=&cam=&tour), Esc steps up.
- Incident panel: nearest cameras (path-weighted), fused situational context
  (ADS-B, AIS, 511NY events, NEXRAD tile sample), units, call spine.
- MapView keeps every layer; adds Camera Viewsheds (approx.) overlay and a
  nearest-camera layer for the selected incident.
- CameraFeed hygiene: hidden-tab/off-screen gating, 2 concurrent HLS cap,
  last good frame + stale badge, frozen-frame signature, HLS stall watchdog,
  provider attribution rendered on the imagery.
- Camera/event popups show source credit, update time and the disclaimer.
- Admin > Camera providers tab with licence status, health, gated confirm.
- Camera detection is not built (blocked on #199/#200); seam only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 17:56:06 -04:00
Logan CusanoandClaude Opus 5.5 23860eb31b Systems: P25 trunked / P25 conventional / analog as one choice (node-26#24)
- system_config.normalize(): a P25 system with config.mode "conventional"
  carries config.channels [{freq MHz, name, nac?}] (1-2, within 1.5 MHz) and
  can no longer carry control channels (422). A missing mode is trunked and
  passes through untouched, so there is no migration. An analog channel is
  mirrored into control_channels for older nodes.
- talkgroups.name_from_system resolves the edge node's synthetic per-channel
  id (freq in 100 Hz units) to the channel name.
- The checkin persists radio_channels (conventional channels + encryption).
- Systems editor: Type = P25 trunked / P25 conventional / Analog (NBFM) /
  DMR, plus a channel-row editor for conventional and a one-frequency form for
  analog. The node page shows conventional channels and an encryption notice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:55:34 -04:00
Logan CusanoandClaude Opus 5.5 7fa50f24d2 approve_node never rotates an existing key; node update-check endpoint; real Update Node button (node-26#14, #15)
- approve_node: reuses node_keys/{id}.api_key when one exists (still
  re-syncs dynsec with it, still never overwrites node_keys). Re-clicking
  Approve can no longer lock a live node out. Reissue stays explicit.
- POST /nodes/{id}/request-update (admin): sets update_requested_at on the
  node doc AND sends MQTT node_update best-effort, so offline/locked-out
  nodes still get it.
- POST /nodes/{id}/update-check (node, inline auth: Bearer api_key for THAT
  node or X-Pickup-Secret): host updater's HTTPS check-in. Returns only
  update_requested_at; records updater_last_check_at / sha / applied result.
  Unknown node -> 401 (no node_id oracle). C2 cannot name a ref, image or
  command.
- Frontend: "Update Node Software" (admin only) calls request-update, honest
  confirm text; node page shows updater check-in and last update state.

Tests: tests/test_node_provisioning.py (12).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 01:09:52 -04:00
Logan CusanoandClaude Opus 5.5 2b42e5ee9a Admin Users: show each user's org and move a user into yours
A viewer whose first login ran self-serve signup got an empty org of
their own (org_role owner), so they saw no incidents or calls, and the
earlier fix deliberately never moved a user who already had an org.
PATCH /admin/users/{uid} now moves a user when org_id is passed
explicitly (claims + org_members, audited with left_org_id; the old org
is not deleted). The user list returns org_id/org_role, and the admin
user panel shows the org and a 'Move to my organization' button when it
isn't yours.

Verified: c2-core pytest 496 passed; frontend tsc --noEmit clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 18:52:55 -04:00
Logan CusanoandClaude Opus 5.5 17da2ff739 Admin-created users join an org, so viewers can actually see data
POST /admin/users set a role claim but never an org_id/org_role claim or
an org_members doc. firestore.rules gates every read on the org_id claim,
so a viewer created from the Users page saw no incidents or calls.

New users now join the requested org (default: the acting admin's own,
else 'founding') as org_role 'member', with the same org_members doc
POST /auth/signup writes. PATCH /admin/users/{uid} attaches an org-less
user the same way, which heals accounts created before this fix; it never
silently moves a user who already has an org.

Verified: c2-core pytest 495 passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 18:16:16 -04:00
Logan CusanoandClaude Opus 5.5 fb2d737d6f Node SDRs: pin OP25 and each service to a dongle by serial (node-26#11)
Pairs with node-26 feat/sdr-pins. NodeRecord gains sdr_pins
(service -> serial), sdr_devices and op25_sdr_serial, mirrored from the
node's checkin. PATCH /nodes/{id} validates pins (known services, one
dongle per service) and sends priority/pins as a 'set_sdr_config'
command, never a config re-push. The node restarts OP25 only when OP25's
own dongle changes. The node page's section becomes 'SDRs' with an OP25
SDR dropdown ('Automatic (first SDR)' + detected dongles) and a
per-service dongle dropdown ('Any spare SDR'), plus duplicate-serial and
double-pin warnings.

Verified: c2-core pytest 490 passed; frontend tsc --noEmit clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 14:36:49 -04:00
Logan CusanoandClaude Opus 5.5 3347ced863 511NY DOT cameras + traffic events map layers (#183)
GET /traffic/511 serves a bbox slice of the statewide 511NY camera and event
feeds from an in-memory cache (cameras 1h, events 2m TTL, fetched lazily) --
public data, so no Firestore writes. A failed refresh keeps the last good data
and reports the error; a schema change (nothing parses) is an error, not an
empty layer. Frontend: opt-in "DOT Cameras" and "Traffic Events" overlays that
fetch only while shown, with an on-map notice when the feed is down or stale.
NY511_API_KEY is optional in config (the API answers without one today; the
terms require a registered key).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 14:26:38 -04:00
Logan CusanoandClaude Opus 5.5 3cbb0828ae Secondary SDR priority: C2 field, node command, dashboard section
Pairs with node-26 feat/secondary-sdr-priority. NodeRecord gains
secondary_sdr_priority (ordered; SDRs beyond OP25's run it top-down) and
secondary_sdr_running, both mirrored from the node's checkin.
PATCH /nodes/{id} accepts the priority, validates it, and sends it as a
'set_secondary_priority' MQTT command. A priority-only change never
re-pushes system config, because that restarts OP25. The node detail page
gets a 'Secondary SDRs' section (admin-editable) with enable, reorder,
save, and live Running / Waiting-for-SDR state from the checkin.

Verified: c2-core pytest 482 passed; frontend tsc --noEmit clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 14:10:11 -04:00
Logan CusanoandClaude Opus 5.5 bff69a1d04 ADS-B map: altitude-colored aircraft icons + click-to-show flight trail
Icons were 16px accent-colored glyphs, indistinguishable from OSM's own
airport symbols. Now a 30px outlined airliner silhouette filled on
tar1090/ADS-B Exchange's altitude hue ramp, with a callsign/altitude
hover tooltip; the selected aircraft grows and gets a white outline.

Clicking an aircraft draws the path heard so far, segment-colored by
altitude. c2-core writes one point per position change to
aircraft/{icao}/positions (deduped in-process, writes now concurrent);
points carry expire_at and a TTL fieldOverride deletes them after ~24h.
Trail reads are gated on the parent aircraft doc's org via get(), so the
query needs no org filter or composite index. The latest stretch without
a 20-min gap counts as the current flight.

Verified: c2-core pytest 479 passed; frontend tsc --noEmit clean (node:20
container on radio-box).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 13:12:36 -04:00
Logan CusanoandClaude Opus 5.5 9f19750ea6 stops review + provisional LLM closure
Replay of 433b35d (server-26#170): traffic stops now open incidents
("45 Adam" stop, "CM2" stop), but the bridge MVA split 33/56 — one
transmission ("transport complete") was read as scene-resolved at 14:44
and an LLM closure was final, so the rest of the MVA opened a new one.

- LLM closure is now provisional (reopenable), like a timer close: it is
  inferred from a single transmission.
- review of 433b35d: backstop only matches a unit's own "on a stop"
  self-report (bare "car stop" mentions and "pull over" dropped), never on
  MTA/rail/bridge/fire/EMS/DPW talkgroups ("Train 4 holding on the stop"),
  negation looks 5 words back, and <=5-word reports ("Adam 3 on a stop")
  get a minimal scene instead of being skipped before the backstop.

c2-core: 471 pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 20:52:31 -04:00
Logan CusanoandClaude Opus 5.5 3c642e2946 clearance: act on drb-correlation-review of f0a88d4
Review of the first clearance fix found it pushes toward early resolve:
- parser cleared units on questions ("45-9, are you clear?"), negations
  ("not clear yet"), orders ("clear the scene", "clear to transport"),
  places/times ("Room 2 clear", "1400 hours, clear") and split "45 9" into
  unit 45. Now rejects "?", not/is/are/you, anything after the status word
  but sign-offs, place/time words; joins "45 9" -> "45-9".
- a clear from a unit never active on an incident was recorded in
  units_cleared and could pass the all-clear gate. Only units actually
  active there can clear there now.
- clearance-only calls skip the LLM tier (same as thin calls): only the
  rules engine's unit match can say which incident a 10-8 belongs to.
- replay incident view carries srcaddr/srcaddrs for the radio-ID clearance
  investigation.

Replay 09-22 10:00-12:00 ET with f0a88d4: real clears 0 -> 2 (both LLM
closure), unit clears still 0 — the parsed clears are right but those
units were never recorded as assigned (Whisper mangles unit IDs at
dispatch), which this commit does not fix.

c2-core: 465 pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 17:26:38 -04:00
Logan CusanoandClaude Opus 5.5 aff3f16d32 Admin Replay: re-run the pipeline over past calls in a sandbox (#170)
Correlation has only ever been measured through live AI windows: days of
wall time per change, and the 09-20→22 window was invalidated outright by
unfunded AI accounts (#169). Recordings are kept regardless of AI, so the
traffic to measure against already exists.

- internal/replay.py: runs a time range of real calls through the live
  pipeline code in original order, clock pinned per call, into
  replay_runs/{run_id}/calls|incidents. Modes: audio (re-transcribe),
  transcripts (re-extract), reuse (correlation only from a prior run's
  scenes). Simulates the idle-resolve and orphan-recorrelation sweeps on
  virtual time. No alerts, summaries, vocab, AI-health alerts or pending
  terms. One run at a time, <=5000 calls, <=7 days.
- firestore.py: ContextVar sandbox redirect for calls/incidents.
- clock.py: ContextVar-pinnable now(), used on the correlation path.
- feature_flags.py: ContextVar flag override so replay runs with live AI off.
- upload.py: scene loop extracted to _extract_and_correlate, shared by the
  live pipeline and replay so replay measures the code that runs live.
- resolved_via on every incident resolve, so a real clear can be told
  from the idle timeout — live and in replay.
- routers/replay.py + /admin Replay tab: estimate, start, compare runs,
  drill into incidents with audio.

Reviewed by drb-correlation-review; its leak and fidelity findings are
fixed and covered by tests. c2-core: 456 pass. Frontend typecheck not run
(no Node on the authoring box).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 15:23:57 -04:00
Logan CusanoandClaude Opus 5.5 40014a47a3 c2-core: Archive "Load more" skipped 150 of every 200 calls
/calls/search scans a 200-row window and returns 50, but the next cursor
was always the last SCANNED row — so with an unfiltered list each page
jumped past the 150 matches it had already read and not shown. Resume
after the last RETURNED row when matches overflow the page; keep the
last-scanned cursor only when the page holds every match (the sparse-
filter case that cursor exists for). Same fix for /calls/eval-queue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 01:05:31 -04:00
Logan CusanoandClaude Opus 5.5 6479174022 frontend: date range picker on Incidents and Archive; fix Archive paging
Incidents and Archive get a from/to date range (native date inputs,
local-day bounds). Incidents filters in the Firestore query; Archive
passes date_from/date_to to GET /calls/search, which applies them as a
started_at range — both ride the existing org_id/started_at index.

Also fixes /calls/search and /calls/eval-queue paging: the cursor went to
Firestore as a raw ISO string against a timestamp field, which compares
by type rather than time, so "Load more" re-read the first page. Cursor
and range bounds are now parsed to datetimes (400 on garbage).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 01:03:40 -04:00
Logan CusanoandClaude Opus 5.5 fa194e0f0a frontend: search, filters and load-more on Incidents; open Archive to viewers
Incidents page: text search (title, location, summary, units, vehicles,
tags, location mentions), status and type filters, and a Load more button
that pages the Firestore query 100 at a time. Filtering runs over the
loaded window, and the page says so when older incidents exist.

Archive (/calls): readable by every org member, not just admins.
GET /calls/search now takes any Firebase token scoped to the caller's org
— the Firestore rules already let members read every call in their org,
so this widens nothing. Attach/detach stays admin-only (UI and routes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 23:53:46 -04:00
Logan CusanoandClaude Sonnet 5 5f85a878fa admin: STT eval harness — record human-verified transcripts, measure real WER (#163)
Build & Deploy / Build & push images (push) Successful in 4m10s
Build & Deploy / Deploy Firestore rules & indexes (push) Failing after 3s
Build & Deploy / Deploy to VM (push) Successful in 1m56s
Build & Deploy / Report a failed deploy (push) Successful in 1s
Backend: three new routes on the calls router, deliberately separate from
PATCH /{call_id}/transcript (a production correction with real side effects
-- re-extraction, incident unlinking, vocabulary learning). This is pure
measurement and must never share that code path.

  GET  /calls/eval-queue        -- calls with a transcript but no
                                    eval_transcript yet, paged (same bounded-
                                    window-plus-cursor shape as /search)
  PUT  /{call_id}/eval-transcript -- records eval_transcript/_by/_at only;
                                      never touches transcript/transcript_corrected
  GET  /calls/eval-stats        -- eval_count + average word error rate of
                                    the raw and corrected machine transcripts
                                    against the human-verified ones

internal/wer.py: standard word-level Levenshtein WER. Returns None (not 0.0)
when the reference is empty -- a call nobody transcribed must not score as a
perfect match.

Frontend: a new "STT Eval" tab on /admin -- one call at a time, audio player,
a textarea pre-filled with the machine transcript to correct into ground
truth, Save & next / Skip, running WER stats at the top. Built for working a
handful of calls at a time over however many sittings it takes, not a
one-shot form: the queue auto-refills from where the last save left off.

Verified: 438 pass, 0 fail (12 new backend tests). Frontend is UNVERIFIED --
this box has no Node.js/npm (confirmed absent), so neither typecheck nor the
dev server could be run. Matches existing code patterns and the CallRecord/
c2api types by manual review only.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-21 00:04:30 -04:00
Logan CusanoandClaude Sonnet 5 1ffff25cd2 upload: let short transcripts (<=5 words) attempt correlation instead of never linking at all
Build & Deploy / Build & push images (push) Successful in 4m8s
Build & Deploy / Deploy to VM (push) Failing after 2m21s
Build & Deploy / Report a failed deploy (push) Successful in 1s
intelligence.py skips GPT extraction for transcripts <=5 words ("10-8", "show
me clear", a unit check-in) -- real cost/hallucination guard, kept as-is. But
upload.py's no-scenes correlation fallback (the path that thin-links a call
by talkgroup even with zero extracted content) excluded ANY skip_reason,
including transcript_too_short -- so this exact population, brief but real
follow-up and clearance traffic, never even attempted to attach to anything.
Found live: "Live, Ossining." sitting an orphan 6 seconds before a real
incident's founding call, on the same talkgroup.

Now only garbage_transcript (Whisper hallucination, no real content) stays
excluded; transcript_too_short reaches the same thin/fast-path fallback
already trusted for no-transcript calls, gated the same way -- same-talkgroup,
recently-active incident required before anything attaches. No GPT re-invoked,
no new cost.

Verified: 410 pass, 0 fail.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-20 16:39:46 -04:00
Logan CusanoandClaude Sonnet 5 3d2b722c64 Wire AIS end to end: telemetry ingestion + live map overlay
node-26#9, same shape as the ADS-B commit. Adds POST /telemetry/ais
(same node-key auth, same org_id-stamped upsert-by-key pattern, this time
by mmsi into a new `vessels` collection) and its docInMyOrg() firestore
rule. Frontend gets useVessels() (mirrors useAircraft(), longer staleness
window since AIS position reports are minutes apart, not seconds) and an
opt-in "Vessels" map overlay.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-20 16:12:56 -04:00
Logan CusanoandClaude Sonnet 5 5537b095df Wire ADS-B end to end: telemetry ingestion + live map overlay
node-26#9. Adds POST /telemetry/adsb (node-key authed via
require_node_service_or_firebase_token) that upserts one Firestore doc per
icao into a new `aircraft` collection, org_id stamped from the reporting
node the same way upload.py defensively stamps `calls`. firestore.rules
gets a matching docInMyOrg()-gated read rule.

Frontend: useAircraft() mirrors useNodes()'s onSnapshot pattern, filtering
docs older than 2 minutes client-side since nothing prunes a stale aircraft
doc server-side yet. MapView gets an opt-in "Aircraft" overlay (unchecked
by default, like the weather radar layer) rendering a rotated plane glyph
per sighting.

Unverified via typecheck — no Node.js/npm on this authoring box yet (see
CLAUDE.md Testing reality). Server side is pytest-covered (test_telemetry.py).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-20 16:08:29 -04:00
Logan CusanoandClaude Sonnet 5 8892e824fc Add second-SDR fields: secondary_sdr_mode + sdr_count on NodeRecord
Server-side half of node-26#9. NodeRecord gains secondary_sdr_mode
(none|adsb|ais|op25_2) and sdr_count; checkin ingestion stores both,
and PATCH /nodes/{id} accepts and re-pushes secondary_sdr_mode the same
way hardware_preset/ppm_override already work, so it isn't wiped by a
system reassignment (see server-26#111 for the pre-existing bug that
pattern avoids repeating).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-20 15:56:33 -04:00
Logan CusanoandClaude Sonnet 5 fb0bb15c22 correlator/intelligence: close the incident-clearance gap (dispatch-to-10-8 lifecycle)
Build & Deploy / Build & push images (push) Successful in 4m10s
Build & Deploy / Deploy to VM (push) Failing after 3m25s
Build & Deploy / Report a failed deploy (push) Successful in 1s
Two independent fixes, found by tracing why incidents never actually close
(only 2/281 incidents in 5 correlation debug windows ever got a non-empty
units_cleared; 183 resolved via the 90-min idle sweep instead of a real clear):

1. Pattern B (re-dispatch accept, no explicit 10-8): reassignment=True already
   fired correctly and suppressed the unit from re-linking to its prior
   incident, but nothing ever released the unit FROM that incident — it just
   sat "active" until the idle sweep timed it out. _release_reassigned_units
   now scans other active incidents for unit overlap on a reassignment and
   clears the unit there, reusing the same units_active/units_cleared merge
   (factored out as _apply_unit_clearance) that explicit 10-8 extraction uses.

2. Pattern A (self-clear) extraction was inconsistent for two reasons: no
   per-system unit ID format awareness anywhere in the pipeline (formats vary
   by department with zero shared convention), and the cleared_units prompt
   rule only accepted a unit self-reporting, missing dispatch confirming a
   unit's status back to them. Added system.unit_format_hint (owner-authored
   free text, GET/PUT /systems/{id}/unit-format, no auto-induction yet) fed
   into the extraction prompt, and broadened the cleared_units rule while
   still requiring an identifiable unit ID (guards against bare "10-8"/"clear"
   noise, including Whisper hallucination runs already caught upstream by
   _is_garbage_transcript).

Verified: 401 pass, 0 fail (local Linux venv ~/venvs/drb-5c — see CLAUDE.md
testing-reality note).

server-26#pending — not yet filed, Gitea unreachable from this sandbox.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-20 14:25:01 -04:00
Logan CusanoandClaude Sonnet 5 8dd636af8f correlator: stop the re-correlation sweep racing an in-flight upload (#131)
server-26#131: same call_id ends up in TWO incidents' call_ids, byte-identical
extracted data, ~2% of linked calls across 3 live dumps. Root cause: the
sweep's orphan filter (incident_id/incident_ids/corr_path all absent) can't
tell 'never processed' apart from 'real-time pipeline is still mid-flight' --
a call whose STT/scene-extraction/correlation chain (routers/upload.py
_run_intelligence_pipeline) hasn't finished yet has none of those fields set,
so the sweep picks it up and correlates it independently, sometimes onto a
different incident than the real-time path lands on. Confirmed in review:
_update_incident/_create_incident append call_id to an incident's call_ids
unconditionally, with no cross-incident dedup guard -- preventing the second
correlation attempt is the only lever available at this layer.

Fix: _run_intelligence_pipeline marks intelligence_started_at on the call doc
before any slow step; the sweep holds back any call whose marker is under 15
minutes old (raised from an initial 5 -- see below), regardless of how
orphaned it otherwise looks. No marker at all (pre-#131 call doc, or the
marker write itself failed) is not held back -- absence isn't evidence of an
in-flight pipeline, and that's #131's own pre-existing population. Also
covers the /calls/{id}/reprocess path, which calls the same
_run_intelligence_pipeline.

15 min, not 5: neither the OpenAI Whisper client nor the Gemini call in
llm_correlator.py sets a request timeout (filed server-26#153), so 5 min was
a guess against an unbounded tail -- drb-correlation-review flagged this.
Raising it is free on the recovery side: a call that finished processing
(linked or genuinely orphaned) always has corr_path set and is already
excluded by the sweep's other filter, so this constant only ever delays
calls that are still actually running. DEFERRED.md row 52 (outside this
repo, Version 5C root) updated to flag its ~6 min timing figure as stale.

recorrelation_sweep.py had zero test coverage before this. New file covers
the guard function's boundary (age < threshold vs exactly-at vs old vs
missing vs unparseable) and one integration-shaped test proving a racing call
never reaches correlate_call while a genuinely-orphaned call still does.

Sandboxed pytest: 381 -> 387.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-14 00:30:24 -04:00
Logan CusanoandClaude Sonnet 5 7717fcccdd correlator/admin: capture per-scene incident_type + severity (#139)
_call_is_substanceless's "type" veto reads ctx["incident_type"] at decision
time, but that value was never persisted per-scene — only the last-scene-wins
flat field, which #138's window-4 dump analysis couldn't tell apart from
cross-scene contamination without re-guessing from a live dump. Adds
incident_type/severity to _apply_and_log's per-scene write and to admin.py's
_scene_summary allowlist (the debug-dump reader has its own field allowlist,
separate from the write side — silently would not have surfaced otherwise).

Sandboxed pytest: 380 -> 381.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-14 00:09:26 -04:00
Logan CusanoandClaude Sonnet 5 b1884852d5 correlator: remove is_dispatch and the tactical fit path entirely (#134)
Full removal, not a hardcoded flag: _is_dispatch_channel, _DISPATCH_TG_RE, the is_dispatch parameter, and _call_fits_incident's tactical branch are gone. One evaluation path for every channel.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-13 14:49:55 -04:00
Logan CusanoandClaude Sonnet 5 c50bfda8db correlator: drop the dispatch/tactical name-guess from the escape hatch, use one window (#115)
Owner correction from direct scanning experience: a talkgroup named tac/tactical only sees materially different traffic during a real incident, and that's rare -- the bulk of traffic on any monitored channel, including high-risk stops and pursuits, runs on the main channel regardless of what it's named. _is_dispatch_channel's string match on the talkgroup name is a naming-convention guess, not a detector of actual channel behavior; trusting it here meant a busy single-channel department not literally named 'dispatch' would silently get the more permissive 15-minute window and could reproduce #115's original bug (the gate never firing on the channels it targets).

Always use tg_dispatch_thin_idle_minutes (5 min) in the escape hatch, regardless of talkgroup name. Does NOT touch incident_correlator.py's own fast/thin idle-window selection, which uses the same dichotomy for a different, decision-changing purpose -- bigger blast radius, left for its own review.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-13 14:07:00 -04:00
Logan CusanoandClaude Sonnet 5 0fe6d3b567 correlator: delete stale scenes on re-extraction instead of leaving them to rot (#96/#114)
Review of #132 found a blocker: PATCH /calls/{id}/transcript wipes tags/severity/location/units/embedding before re-extraction, but not the new scenes map, and doc_set(merge=True) can only add/overwrite nested map keys, never remove one. A call corrected from 3 scenes to 1 kept scenes.1/scenes.2 with pre-correction transcripts and incident_ids forever -- corrupting the exact per-scene tally #96 exists to make trustworthy, and able to re-feed stale text into #114's summarizer fix if a stale scene's incident_id still names a real incident.

Fix: fstore.doc_update(...,{"scenes": fstore.DELETE_FIELD}) -- a real delete, not a merge over an empty map. Added fstore.DELETE_FIELD (re-exports the real firebase_admin sentinel) and stubbed it in the sandboxed test conftest, which didn't have it. Also softened an overclaiming docstring: the Firestore nested-merge behavior is verified against the doc_set wrapper's pass-through, not against live Firestore.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-13 13:33:52 -04:00
Logan CusanoandClaude Sonnet 5 fae84a45c3 correlator+summarizer: per-scene call-doc storage, fixes #96 and #114's real fix
Every scene of a multi-scene call correlates independently in upload.py's
scene loop, but every scene's corr_debug was written flat onto the same
shared call doc — scene 2's write silently clobbered scene 1's
corr_path/corr_consensus/etc (#96), and summarizer.py read the whole call's
raw transcript per linked call, mixing text from scenes the incident had
nothing to do with, while ignoring transcript_corrected entirely (#114).

Fix: thread a scene_index from both `for scene in scenes:` loops in
upload.py down through _correlate_with_consensus ->
incident_correlator.preview_correlation/correlate_call -> _build_context ->
ctx["scene_index"]. incident_correlator._apply_and_log now writes, in the
same Firestore call:
  - the existing flat corr_* fields, unchanged (last-scene-wins, the safe
    backward-compatible default for any reader that doesn't know about
    `scenes` yet)
  - a new nested `scenes.<scene_index>` entry with {transcript, incident_id,
    corr_debug}, via doc_set(..., merge=True). Firestore's
    DocumentReference.set(data, merge=True) recursively merges nested map
    fields by key (documented SDK behaviour, not assumed) — a write to
    scenes.1 merges alongside an existing scenes.0 instead of replacing the
    whole `scenes` map.
scene_index defaults to 0 for every caller with no scene concept (the
recorrelation sweep, the no-scenes-extracted orphan-check path), so a plain
single-scene call still gets a one-entry `scenes` map equivalent to reading
its flat fields today.

admin.py's _call_summary exposes the new `scenes` list per call (each entry
carrying the same corr_* field names as the flat fields, so the two shapes
are interchangeable to the tally); the summary tally now iterates each
call's scenes-if-present, else its own flat fields, so a 2-scene call with
two different corr_path values counts as two data points instead of one
blend. New `scene_decision_count` sits next to `linked_call_count` to make
that distinction visible.

summarizer.py's _scene_text_for_incident reads a linked call's `scenes` map
to find the scene(s) whose corr_debug recorded a link into the specific
incident being summarized, joining more than one if several scenes landed
in the same incident. Falls back to transcript_corrected-or-transcript for a
call doc with no `scenes` field (predates this change) — the one-liner half
of #114, worth doing regardless since it stops raw-transcript summaries even
for old-schema docs.

Does not touch #80/#95/#102's existing ctx-threading fixes
(embedding/severity/coords/LLM-prompt-transcript) — correct as-is, out of
scope here.

Tests: 14 new (test_per_scene_call_doc.py, test_summarizer_scene_transcript.py,
additions to test_admin_debug_correlation.py) covering the merge shape,
last-scene-wins flat-field backward compat, the admin tally's per-scene vs
per-call counting (including old-schema fallback), and the summarizer's
scene-specific text selection (including old-schema fallback). Full sandboxed
suite: 364 -> 378 passed, all green.

Fixes server-26#96, server-26#114

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-13 13:25:37 -04:00
Logan CusanoandClaude Sonnet 5 3ae0bb2d5b intelligence: run the chatter classifier before the too-short skip, not after (#127)
82% of the classifier's backtest flags were <=5-word transcripts that already exit at skip_reason=transcript_too_short before the classifier ever ran, so shadow mode was on track to observe roughly a fifth of the real catch rate. Compute the verdict once, ahead of that check, and fold it into whichever doc_set already runs (no extra Firestore write). Also add a chatter_classifier_flagged/reason tally spanning both linked calls AND orphans in admin.py's summary block -- the target population is non-events, which land as orphans or single-call incidents, so linked alone undercounts it the same way corr_gate_veto would have without the #126 fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-13 12:42:44 -04:00
Logan CusanoandClaude Sonnet 5 05ddec8284 intelligence: shadow-mode upstream dispatch-vs-chatter classifier (server-26#127)
Three live measurement windows and two consensus-layer fixes (#125, #126)
converged on one decision (CORRELATION_REVIEW_0907b.md, _0912.md): stop
iterating the correlator's consensus layer, the actual lever is upstream —
a classifier in scene extraction that recognizes radio housekeeping (roll
call, bare 10-4/10-8/98 acknowledgements, unit check-ins) before it ever
becomes a scene for the correlator to judge.

Adds app/internal/chatter_classifier.py: a pure classify_chatter(transcript)
function recognizing two shapes drawn from hand-labeled examples in the
review docs, cross-referenced against the real dumps — not invented regexes.
Deliberately conservative: anything that doesn't cleanly reduce to a known
shape returns (False, None) and the existing pipeline runs unchanged.

SHADOW MODE ONLY. intelligence.extract_scenes computes the verdict next to
the existing _is_garbage_transcript / transcript_too_short gates and writes
chatter_classifier_verdict / chatter_classifier_reason onto the call doc,
but does not skip extraction. admin.py's correlation-debug _call_summary
surfaces both fields, same pattern as corr_gate_veto (#115/#126), so the
next live window can measure the real-world false-positive rate before
anything is wired to actually skip extraction. TODO(server-26#127) marks
the call site.

Backtest against all three existing dumps (1002 calls): 154 flagged, 0
false positives (no flagged call carries tags, coords, non-routine severity,
or matches any review-doc-named dangerous-to-drop transcript — the major
extinguishing-fire call, geocoded calls, pursuit updates, the Pelham Station
subject check, the property-retrieval call, all individually verified).

tests/test_chatter_classifier.py: real transcripts from the dumps/review
docs in both directions. Sandboxed pytest 332 -> 364, green.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-12 23:59:53 -04:00
Logan CusanoandClaude Sonnet 5 83beb2bf35 admin: surface corr_gate_veto on the correlation-debug endpoint (#115)
corr_gate_veto was written to corr_debug but the admin endpoint's whitelist (_call_summary + the summary tally) never surfaced it, so the last commit's whole point -- measuring window #4 instead of guessing -- would have produced nothing to read. Add it to both. Also softened the docstring's remaining overclaim: whether the active-only ctx[recent] limitation explains the 2/24 window-3 misses is unanswered, not confirmed -- read corr_gate_veto next window instead of asserting a guess again.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-12 04:47:15 -04:00
Logan CusanoandClaude Sonnet 5 598054746a correlator: mirror the dispatch/tactical idle split in the escape hatch, record the gate-veto reason (#115)
Review of #126 found: (1) the escape hatch applied tg_dispatch_thin_idle_minutes (5 min) unconditionally, but incident_correlator's own fast/thin path only uses that on dispatch channels and 15 min on tactical ones via _is_dispatch_channel -- mirrored the same selection here, plus a config.py note flagging the second consumer. (2) the docstring claimed a 'confirmed explanation' for 2 window-3 gate misses that was actually wrong (self-contradictory in its own text); replaced the guess with corr_gate_veto, written into corr_debug on every llm=orphan/rules=new disagreement that escalates, so window #4 can see *why* each one escaped instead of reconstructing it from the raw dump.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-12 04:42:14 -04:00
Logan CusanoandClaude Sonnet 5 400b74b519 correlator: shrink the same-talkgroup escape hatch from 2h to a few minutes (#115)
_recent_incident_on_same_talkgroup previously treated ANY same-talkgroup
incident within the 2-hour correlation_window_hours lookback as 'recent',
which disabled the whole LLM-orphan consensus gate on busy dispatch
channels: window #3 (CORRELATION_REVIEW_0912.md) measured 0/24 gate fires
against the exact target shape (rules=new, llm=orphan, tiebreak=new), with
22/24 explained by a same-talkgroup incident existing somewhere in the
prior 2h — nearly guaranteed on channels producing 3-13 incidents/2h.

Now the escape hatch only counts an incident as recent within
settings.tg_dispatch_thin_idle_minutes (5 min), reusing the same recency
bound the fast/thin path already uses for the 'dispatch, thin ack 10-30s
later' case this hatch exists for, instead of inventing a new constant.

Investigated the 2 unexplained misses (no same-tg incident found even by
a naive full-collection timestamp scan): confirmed ctx["recent"] is built
from status=="active" incidents with over-capacity incidents dropped
(_build_context / _drop_capped), not a full collection scan — an incident
that has auto-resolved or hit incident_max_calls/incident_max_duration
within the window is invisible to this check even though it is
chronologically recent. This does not explain the 2 misses (a same-tg
incident was absent by both checks there, so some other
_call_is_substanceless condition must be responsible), but it is a real
gap in the check as written. Documented in the docstring with a
TODO(server-26#115); fixing it needs a new, non-active-filtered Firestore
query, out of scope for this pass.

Tests: added a regression test proving an incident inside the old 2h
window but outside the new 5-minute window now correctly gates (fails on
main, passes here), plus a test proving a truly recent (<5min) same-tg
incident still escapes the gate as intended. Sandboxed pytest: 327 -> 329
passed (2 new tests), all green.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-12 04:28:07 -04:00
Logan CusanoandClaude Sonnet 5 15a9d10666 correlator: keep the tiebreak for typed / reassignment calls in the orphan gate (#115)
_call_is_substanceless mirrored has_event_substance but not the creation gate's type-resolved short-circuit, so a routine-severity fire/medical call with no coords/tags/vehicles — or a reassignment (unit pulled to a new job) — could be gated to orphan where rules would open an incident. Bail out of the gate on incident_type or reassignment.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-07 23:57:20 -04:00
Logan CusanoandClaude Sonnet 5 dd426572fc correlator: fix consensus orphan-gate to test call substance, not empty corr_debug (#115)
The gate added in ca1d8fb checked rules_decision["corr_debug"] for a positive
signal, but that dict is empty at preview time for action=="new" (corr_path is
written at apply time). The check was always False, so the gate fired on real
events — replayed against corr_dump_9-7_pm.json it dropped ~36 linked calls
including a major "extinguishing fire", a moderate fire-alarm, geocoded calls
and pursuit updates.

Gate now runs against ctx (fully populated at preview time). It fires ONLY when
the call is substanceless: routine severity, no vehicle/geocode/tag, and no
incident already running on the same talkgroup. Any of those escalates to the
tiebreak instead. The substance predicate (has_event_substance) is factored out
of incident_correlator's creation gate and shared, so the two cannot diverge.

recorrelation_sweep: a call the gate parked gets a longer link-only retry budget
(10 vs 3) — the gate fires before any incident for the job exists, so the
substantive call that justifies linking can land after the standard ~6 min.
Still create_if_new=False.

incident_correlator location path: evaluate every in-radius candidate and link
the nearest that carries corroboration, instead of the first in an unsorted
`recent`. A unit-overlap location link is now tagged "location_unit_overlap" so
it stops merging into the fast path's bucket in the admin fit-signal histogram.

tests/test_consensus_gate.py: replaced the corr_debug-signal cases with ctx
substance cases (severity, coords, tags, vehicles, same-tg incident); added a
nearest-wins location test; the two location guard tests now assert they reach
the new guard. Full drb-c2-core suite 322 -> 325.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-07 23:50:41 -04:00
Logan CusanoandClaude Sonnet 5 ca1d8fbdae correlator: gate LLM-orphan against rules-new instead of escalating to tiebreak (#115)
CORRELATION_REVIEW_0907b.md measured that radio housekeeping (unit
check-ins, roll call, 10-8/10-98 clearings) is being promoted to
incidents. Every case reads corr_llm_action=orphan, corr_rules_action=new,
corr_consensus=tiebreak -> new: the cheap LLM correctly reads "not an
incident", the rules engine says `new` only because there is no incident to
link to, and the smart tiebreaker then sides with rules ~21/21. Reframing
the tiebreaker prompt (#116) did nothing. The fix is a consensus-logic gate,
not another prompt.

Fix 1 (routers/upload.py) - LLM-orphan gate in _correlate_with_consensus:
when the cheap LLM says `orphan` and the rules engine says `new` with NO
positive event signal, resolve to `orphan` and skip the tiebreak call
entirely. "No positive signal" = the rules corr_debug carries neither a
positive corr_path (unit-continuity / location / fast/disambig / fast/single)
nor a positive corr_fit_signal (unit_overlap / location_proximity). When it
does carry one, the existing escalation-to-tiebreak is kept so a genuine
event the LLM misreads as orphan still gets the second look. The resolved
outcome records corr_consensus="llm_orphan_gate" (greppable, distinct from
"tiebreak") and keeps corr_llm_reasoning / corr_rules_action /
corr_llm_action populated.

Fix 2 (incident_correlator.py) - tighten corr_path=location: the location
path linked on a bare sub-location_proximity_km (0.5 km) distance with no
unit or content check, which stitched a vehicle lockout to a station-restroom
slip and merged two different churches an hour apart. A location link now
requires unit overlap with the candidate OR a distance under a tighter bar
(_LOCATION_TIGHT_PROXIMITY_KM = 0.2 km). Pursuit incidents keep their
movement-speed-validated wide radius. A surviving location link now also
writes corr_fit_signal (unit_overlap | location_proximity), consistent with
Fix 1's positive-signal set.

Tests: new tests/test_consensus_gate.py (13 cases) - the gate resolves to
orphan without calling tiebreak on a no-signal disagreement; a unit_overlap /
location_proximity / unit-continuity / fast-disambig rules signal still
escalates; llm=link vs rules=new still escalates; the location path drops a
shared-area candidate with neither unit overlap nor tight proximity, links on
unit overlap, and links on tight proximity alone. Full c2-core suite
309 -> 322 passing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
2026-09-07 23:32:52 -04:00
Logan CusanoandClaude Sonnet 5 ef1e3d7f9d correlator: LLM tier reads the scene's transcript, not the whole call (server-26#102)
The last leg of the #80/#95 scene-context leak. llm_correlator._call_block
read call_doc's whole-call transcript for every scene, so on a multi-scene
call every scene's cheap-tier and tiebreaker decision was made against text
that also contained the other scenes.

- intelligence.py: each processed[] scene now carries its own "transcript" —
  transcript_corrected, else this scene's segments joined, else (single scene)
  the whole transcript.
- _build_context / preview_correlation / correlate_call: take a `transcript`
  param; _build_context resolves ctx["scene_transcript"] from it, falling
  back to the call doc (sweep, single-scene, tests) — the fallback is kept
  here, unlike embedding/severity, because a scene always has real text.
- upload.py: both scene loops pass scene["transcript"].
- llm_correlator._call_block: reads ctx["scene_transcript"] (call-doc
  fallback retained for test-built ctx).
- recorrelation_sweep: passes the call doc's text explicitly.
- +1 regression test. Full c2-core suite green (296 passed, sandboxed venv).

NOT for merge until the running correlation measurement window closes and its
dump is analysed — deploying a correlator change mid-window would mix old and
new behaviour in the sample.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-07 00:06:37 -04:00
Logan CusanoandClaude Sonnet 5 a9d1d2475a correlator: judge each scene on its own embedding + severity (server-26#80, #95)
intelligence.py writes only the primary scene's embedding and severity to
calls/{id}. _build_context read them back off the call doc, so every
non-primary scene of a multi-scene call was correlated against scene 1's
semantic vector and severity rung: a scene about a different event scored
on the embedding path against the wrong incident, and could inherit a
minor/moderate/major severity it never had, clearing the creation gate on
borrowed weight. Same defect and same fix as the #87 coords leak.

- _build_context / preview_correlation / correlate_call: take embedding and
  severity as params; drop the call_doc.get() fallbacks. A scene that
  passes none has none, and is judged thin on its own signal.
- upload.py: both scene loops pass scene["embedding"] / scene["severity"];
  _correlate_with_consensus forwards them. The no-scene unclassified branch
  passes neither (correct: no scene, judged thin).
- recorrelation_sweep: passes the call doc's stored values explicitly
  (whole-call re-link, link-only, so a borrowed severity cannot create).
- intelligence.py: SCENE DETECTION prompt tightened toward one scene
  (server-26#5, partial) - MULTIPLE only for genuinely separate events,
  "when unsure, one scene", plus a not-a-new-scene list.
- test_incident_identity.py: +2 regression tests mirroring the #87 test.

Full c2-core suite green (295 passed). #5 prompt change is unmeasured -
needs a scoped correlation-only window. Known remaining legs, tracked
separately: llm_correlator._call_block still reads the whole-call
transcript per scene; content-divergence veto skips on a None embedding.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-06 13:01:59 -04:00
Logan CusanoandClaude Opus 5 8b6c170265 Close viewer-triggerable OpenAI spend on incident summarize (server-26#81)
Build & Deploy / Build & push images (push) Failing after 1m58s
Build & Deploy / Deploy to VM (push) Skipped
Build & Deploy / Report a failed deploy (push) Successful in 2s
POST /incidents/{id}/summarize was gated by require_service_or_firebase_token,
which accepts any authenticated Firebase user including role "viewer". That
route spends OpenAI credits via the background summarizer. The call-side
equivalent was already moved to require_admin_token; this brings the incident
side in line with it.

The frontend's two "summarize now" buttons on the incident detail page are
already gated behind isAdmin, so this backend change matches existing UI
behavior exactly and does not break any viewer/operator surface — it only
closes direct-API access for non-admins.

Swept every other route in incidents.py: list/get are reads with no spend and
correctly stay open to any signed-in user; create/update/delete/link/unlink
were already require_admin_token. No other sibling route needed changing.

Adds test_incident_summarize_auth.py pinning the dependency wiring directly
(the convention used in test_admin_feature_flags.py), so a future revert back
to the weak dependency fails a test immediately.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-01 02:45:08 -04:00
Logan CusanoandClaude Opus 5 865b5b4317 Close the /admin/features side-door that needed a container shell to flip AI spend
Build & Deploy / Build & push images (push) Successful in 4m15s
Build & Deploy / Deploy to VM (push) Successful in 1m56s
Build & Deploy / Report a failed deploy (push) Skipped
Board minutes #62 Decision 2 (server-26#64), due 2026-08-31. CTO draft #60
finding 1 and CISO draft #61 finding 3 reached this independently.

GET/PUT /admin/features accepted only a Firebase admin token, so the unattended
runbook had no headless path and SSHed into the c2-core container to write
config/ai_features with the admin SDK. Moving a platform-wide AI cost switch
required a full container shell, and set_flags() wrote no audit entry either
way, so a flag flip was unattributable however it happened.

- New agent_service_key (AGENT_SERVICE_KEY), deliberately separate from the
  Discord bot's service_key. Sharing one key would collapse two principals into
  a single unattributable identity in every log line, and the bot has no
  business flipping AI flags regardless.
- require_agent_key_or_admin accepts the agent key or a Firebase admin, and
  rejects the Discord key. The "key is configured" guard is load-bearing:
  compare_digest("", "") is a match, so a deployment that never set the key
  would otherwise accept an empty credential.
- set_flags() writes an audit_log entry with before/after values and the actor,
  wrapped so an audit failure cannot lose the flag write or 500 the route.
- Cascade helper sets the global doc and every system carrying an ai_flags
  override in one call. A global False already beats everything, but a system
  False beats a global True, so turning AI *on* could half-apply and leave a
  radio system hot after shutoff. It scans for the override rather than
  hardcoding the two known system IDs, so a new system cannot silently defeat
  it.
- cascade defaults to False. PUT /systems/{id}/ai-flags and the AiFlagsPanel
  toggle mean a per-system override is deliberate operator intent; cascading by
  default would erase it on any unrelated global flip. The runbook opts in.

Issue items 5 and 6 (retiring the SSH path from drb-worksession.md) are NOT
done here and the runbook is untouched. The credential does not exist in
production yet, so the SSH path is still the only one that works; retiring it
now would break the next unattended run. Owner activation is recorded on #64.

Tests 273 -> 289.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-30 02:52:00 -04:00
Logan Cusano d18e4f0743 Make "AI is off" true, and stop the transcript PATCH from destroying calls
Build & Deploy / Build & push images (push) Successful in 4m2s
Build & Deploy / Deploy to VM (push) Successful in 1m53s
Build & Deploy / Report a failed deploy (push) Skipped
config/ai_features was not the switch it was documented to be. Three paths
spent money with it off, and one path read it wrong, so per-system opt-outs
did not opt anything out.

- Correlation in the ingest pipeline tested the raw global flag instead of the
  per-system resolution. With a system opted out, extraction was skipped but
  the no-scenes fallback still correlated the call with empty tags, taking the
  thin/recency path and attaching it to whatever incident was most recent on
  that system. The opt-out did not disable correlation, it disabled good
  correlation and left the worst kind running. (#75)

- Transcript correction ran on every transcribed call gated only by an env var,
  spending Gemini tokens and a Places lookup per proposed location. An
  "STT-only" window was never STT-only and its cost could not be attributed.
  Now behind transcript_correction_enabled. (#76)

- _run_extraction_pipeline and the vocabulary learner, both reachable from
  PATCH /calls/{id}/transcript, checked no flags at all. (#76, #81)

The flag resolver now lives in feature_flags.resolve_flags() rather than as a
local helper in upload.py. Three copies of that logic is how #75 happened.

PATCH /calls/{id}/transcript now refuses with 409 when correlation is off.
That route wipes tags, severity, location, units, embedding and unlinks the
call from every incident before queueing re-extraction. Gating extraction
alone would have made it destructive-only in the standing flags-off
configuration: the call left blank and orphaned forever, with the route still
answering 200. The wipe and the rebuild are one transaction in intent, so it
refuses before the first write.

Also: the summarizer's stale-incident sweep is no longer behind
summaries_enabled. It is pure Firestore with no model call in it, and gating
it meant nothing auto-resolved while AI was off - so every incident stayed
active forever and the candidate set every correlation reads kept growing.

transcript_correction_enabled is documented as NOT a pure cost lever. The
corrector is also the noise gate that sets not_speech; with it off, recogniser
noise reaches extraction as a real transcript, comes back thin, and
auto-attaches. Never open an evaluation window with correction off and
correlation on.

14 tests added covering flag precedence, both pipeline paths, the 409, the
correction gate and the summarizer no-op. Suite: 264 passed.

Refs #75, #76, #81, #45.
2026-08-27 02:49:09 -04:00
Logan CusanoandClaude Opus 5 964343c819 area_context v2 + Maps place verification (server-26#36, #37)
Build & Deploy / Build & push images (push) Successful in 4m9s
Build & Deploy / Deploy to VM (push) Successful in 1m54s
Build & Deploy / Report a failed deploy (push) Skipped
#36 — the correction pass shipped in 58efdbd was right, its reference-data
shape was not. One shape now, at both scopes, every field nullable:

  area_context: { municipality?, county?, state?,
                  center?, radius_km?, resolved_from?, resolved_at?,
                  local_knowledge?: [{term, meaning}] }

`state` closes the ambiguity that made "Ossining" a national guess.
`local_knowledge` replaces roads[]/landmarks[], which could not hold
intersections, schools or nicknames and carried no meanings — `11-X-ray` is
useless alone, `11-X-ray — MTA PD patrol unit` is what a corrector can act on.
Pre-#36 roads[]/landmarks[] are read forward as bare terms so nothing an
operator already entered is lost.

Nullability is the mechanism: which scope gets filled is the operator's
declaration of how homogeneous the system is. One town — fill it once at system
level. Statewide — leave it blank and fill each talkgroup.

The backend owns the derived anchor. PUT /systems/{id} merges config.talkgroups[]
against what is stored instead of writing the client's blob verbatim, which
would have erased the anchor and the pending queue — the same defect as the
ten_codes wipe.

#37 — Maps as a verifier, not as prompt stuffing. The corrector emits its
location nouns; each is geocoded against the talkgroup's anchor, and on a miss
we look for a sound-alike that does resolve there, correct to it, and propose
{term, meaning} to that talkgroup. Cost scales with location nouns, not calls.

No anchor means SKIP. An area too wide to discriminate stores no anchor at all,
because a statewide radius would confirm anything inside it — verification that
passes everything is worse than none, since it reads as a check in the data.

Also re-anchors _geocode_location, which rejected results >40km from the NODE
(server-26#6). An antenna is not a jurisdiction; distance-from-node was always
a stand-in for the anchor and is now only the fallback.

The induction loop proposes at talkgroup level and never promotes. Blast
radius: a wrong term on a channel misleads that channel, the same term
system-wide misleads one 400km away on a statewide system.

38 new tests; 240 pass. Frontend typechecks clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-23 16:43:59 -04:00