correlator: shrink the same-talkgroup escape hatch from 2h to a few minutes (#115) #126

2026-09-12T04:31:07-04:00

logan commented

2026-09-12 04:31:07 -04:00

Fixes the escape hatch that let PR #125's LLM-orphan gate fire 0/24 times in live traffic (window #3, CORRELATION_REVIEW_0912.md): _recent_incident_on_same_talkgroup treated ANY same-talkgroup incident in the last 2 hours as "recent," which a busy dispatch channel satisfies almost unconditionally.

3 commits, 2 review rounds:

400b74b — shrink the window from 2h to settings.tg_dispatch_thin_idle_minutes (5 min), matching the fast/thin path's own bound for the same "dispatch, thin ack" case.
5980547 (round-2 fixes) — the 5-min constant was dispatch-only correct but applied unconditionally; now mirrors incident_correlator._is_dispatch_channel to pick 5 min (dispatch) vs 15 min (tactical), same as the fast/thin path does. Also: _call_is_substanceless now returns (bool, reason) and the reason is written to corr_debug["corr_gate_veto"] on every llm=orphan/rules=new escalation, so the next measurement window reads why each call escaped instead of reconstructing it by hand (a first attempt at that reconstruction produced a self-contradictory wrong answer for 2/24 misses — removed that claim).
83beb2b — corr_gate_veto was written to the call doc but the /admin correlation-debug endpoint's whitelist never surfaced it, so window #4 would've had nothing to read. Added to _call_summary and the summary tally.

Expectation for window #4 (not a defect, not fixed here): the escape hatch still has no single-candidate requirement, unlike the fast/thin tier-2 it mirrors (which voids itself past 1 candidate). With 6-13 active same-tg incidents typical on tg9048, expect partial recovery of the 24 target calls, not all of them — that's the next window's decision, informed by the new corr_gate_veto tally.

Sandboxed pytest: 327 → 332, all green (test_consensus_gate.py 16→22 cases, including a dispatch-vs-tactical split test at the exact 8-minute boundary — config.py's own tuning note records real wrong-attach cases at ≥8.2 min idle, not an arbitrary number).

🤖 Generated with Claude Code

Fixes the escape hatch that let PR #125's LLM-orphan gate fire 0/24 times in live traffic (window #3, `CORRELATION_REVIEW_0912.md`): `_recent_incident_on_same_talkgroup` treated ANY same-talkgroup incident in the last 2 hours as "recent," which a busy dispatch channel satisfies almost unconditionally. 3 commits, 2 review rounds: 1. **`400b74b`** — shrink the window from 2h to `settings.tg_dispatch_thin_idle_minutes` (5 min), matching the fast/thin path's own bound for the same "dispatch, thin ack" case. 2. **`5980547`** (round-2 fixes) — the 5-min constant was dispatch-only correct but applied unconditionally; now mirrors `incident_correlator._is_dispatch_channel` to pick 5 min (dispatch) vs 15 min (tactical), same as the fast/thin path does. Also: `_call_is_substanceless` now returns `(bool, reason)` and the reason is written to `corr_debug["corr_gate_veto"]` on every `llm=orphan`/`rules=new` escalation, so the *next* measurement window reads why each call escaped instead of reconstructing it by hand (a first attempt at that reconstruction produced a self-contradictory wrong answer for 2/24 misses — removed that claim). 3. **`83beb2b`** — `corr_gate_veto` was written to the call doc but the `/admin` correlation-debug endpoint's whitelist never surfaced it, so window #4 would've had nothing to read. Added to `_call_summary` and the summary tally. **Expectation for window #4 (not a defect, not fixed here):** the escape hatch still has no single-candidate requirement, unlike the fast/thin tier-2 it mirrors (which voids itself past 1 candidate). With 6-13 active same-tg incidents typical on tg9048, expect partial recovery of the 24 target calls, not all of them — that's the next window's decision, informed by the new `corr_gate_veto` tally. Sandboxed pytest: 327 → 332, all green (`test_consensus_gate.py` 16→22 cases, including a dispatch-vs-tactical split test at the exact 8-minute boundary — `config.py`'s own tuning note records real wrong-attach cases at ≥8.2 min idle, not an arbitrary number). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

logan added 1 commit 2026-09-12 04:31:07 -04:00

correlator: shrink the same-talkgroup escape hatch from 2h to a few minutes (#115 ) 400b74b519

_recent_incident_on_same_talkgroup previously treated ANY same-talkgroup
incident within the 2-hour correlation_window_hours lookback as 'recent',
which disabled the whole LLM-orphan consensus gate on busy dispatch
channels: window #3 (CORRELATION_REVIEW_0912.md) measured 0/24 gate fires
against the exact target shape (rules=new, llm=orphan, tiebreak=new), with
22/24 explained by a same-talkgroup incident existing somewhere in the
prior 2h — nearly guaranteed on channels producing 3-13 incidents/2h.

Now the escape hatch only counts an incident as recent within
settings.tg_dispatch_thin_idle_minutes (5 min), reusing the same recency
bound the fast/thin path already uses for the 'dispatch, thin ack 10-30s
later' case this hatch exists for, instead of inventing a new constant.

Investigated the 2 unexplained misses (no same-tg incident found even by
a naive full-collection timestamp scan): confirmed ctx["recent"] is built
from status=="active" incidents with over-capacity incidents dropped
(_build_context / _drop_capped), not a full collection scan — an incident
that has auto-resolved or hit incident_max_calls/incident_max_duration
within the window is invisible to this check even though it is
chronologically recent. This does not explain the 2 misses (a same-tg
incident was absent by both checks there, so some other
_call_is_substanceless condition must be responsible), but it is a real
gap in the check as written. Documented in the docstring with a
TODO(server-26#115); fixing it needs a new, non-active-filtered Firestore
query, out of scope for this pass.

Tests: added a regression test proving an incident inside the old 2h
window but outside the new 5-minute window now correctly gates (fails on
main, passes here), plus a test proving a truly recent (<5min) same-tg
incident still escapes the gate as intended. Sandboxed pytest: 327 -> 329
passed (2 new tests), all green.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix

logan referenced this pull request

2026-09-12 04:31:19 -04:00

Consensus tiebreaker manufactures incidents — resolves ~all disagreements as "new", overriding the LLM's "link" #115

logan referenced this issue from a commit

2026-09-12 04:42:17 -04:00

correlator: mirror the dispatch/tactical idle split in the escape hatch, record the gate-veto reason (#115)

logan added 1 commit 2026-09-12 04:42:17 -04:00

correlator: mirror the dispatch/tactical idle split in the escape hatch, record the gate-veto reason (#115 ) 598054746a

Review of #126 found: (1) the escape hatch applied tg_dispatch_thin_idle_minutes (5 min) unconditionally, but incident_correlator's own fast/thin path only uses that on dispatch channels and 15 min on tactical ones via _is_dispatch_channel -- mirrored the same selection here, plus a config.py note flagging the second consumer. (2) the docstring claimed a 'confirmed explanation' for 2 window-3 gate misses that was actually wrong (self-contradictory in its own text); replaced the guess with corr_gate_veto, written into corr_debug on every llm=orphan/rules=new disagreement that escalates, so window #4 can see *why* each one escaped instead of reconstructing it from the raw dump.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix

logan added 1 commit 2026-09-12 04:47:18 -04:00

admin: surface corr_gate_veto on the correlation-debug endpoint (#115 ) 83beb2bf35

corr_gate_veto was written to corr_debug but the admin endpoint's whitelist (_call_summary + the summary tally) never surfaced it, so the last commit's whole point -- measuring window #4 instead of guessing -- would have produced nothing to read. Add it to both. Also softened the docstring's remaining overclaim: whether the active-only ctx[recent] limitation explains the 2/24 window-3 misses is unanswered, not confirmed -- read corr_gate_veto next window instead of asserting a guess again.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix

logan referenced this pull request

2026-09-12 04:47:41 -04:00

Consensus tiebreaker manufactures incidents — resolves ~all disagreements as "new", overriding the LLM's "link" #115

logan merged commit 11c98daed0 into main

2026-09-12 04:47:50 -04:00

logan deleted branch fix/115-escape-hatch-window

2026-09-12 04:47:50 -04:00

logan referenced this pull request

2026-09-12 23:58:06 -04:00

Upstream dispatch-vs-chatter classifier — shadow mode #127

logan referenced this issue from a commit

2026-09-13 00:00:02 -04:00