From the 2026-09-07 correlation measurement window (CORRELATION_REVIEW_0907.md, corr_dump_9-7_0437am.json). This — not scene detection (#5) — is the dominant over-split driver.
tiebreak ran on 40 / 88 calls (45%) — nearly every call hits the expensive smart model. Large Gemini spend on its own.
Cheap LLM wanted link → tiebreaker chose "new" 8/10.
Cheap LLM wanted orphan → tiebreaker opened a new incident 24/25.
32 of 55 window "new" incidents came out of a tiebreak where the cheap LLM wanted to link or orphan.
The tiebreaker's own _RULES (shared in llm_correlator.py) say "orphan when in doubt — conservative is always correct" and "new only if the call has a clear incident_type AND describes a distinct, identifiable scene." Its behaviour is the opposite.
Concrete cases
f53119a5 — same TG-383 event ("female backed up the ramp and left", 05:12–05:16) landed in two incidents (daa58459 "Vehicle Leaving Scene", e55de831 "Vehicle Check at ramp from Midland"). corr_consensus: tiebreak, corr_llm_action: link → corr_path: new + double-linked.
Mohegan Park Ave (TG 9560): 4e4a7864 "Nuisance Alarm" (07:54:57) + 0a95dc40 "Noise Complaint" (07:55:52) — same car-alarms call 55s apart. 2nd call corr_llm_action: link → corr_path: new.
Where to look
llm_correlator.tiebreak() + _build_tiebreak_prompt() — is the prompt biased toward "new"? Does it actually instruct the model that resolving a link/new disagreement toward link is the conservative choice for an over-splitting system?
_inc_summary() — the active-incident block the tiebreaker sees is id | type | loc | units[:6] | tags[:4] | idle. No transcript, no call list. That may be too thin for the model to justify a link, so it defaults to "new".
decisions_agree() / the consensus gate in upload.py:_correlate_with_consensus — 40/88 going to tiebreak suggests the cheap tier and rules disagree far more than expected; check whether the cheap LLM is being fed enough to agree with an obvious link.
Interaction with #96: corr_debug is per-call last-write-wins, so for multi-scene calls these fields are a splice. The single-scene majority makes the aggregate directionally sound but a per-scene record (#96) is needed before tuning against exact numbers.
Related
Geocode coverage is near-zero (location_proximity fired 5× in 154 calls), so the location-fit path can't back up a link — "Ave" vs "Avenue" splits. Worth a normalisation pass (St/Street, Ave/Avenue, Rd/Road) either as its own issue or folded here.
#103 (unit-overlap veto dark on None embedding), #86 (0.82 veto literal) — same bounding problem from the rules side.
#5 (scene over-split prompt) — measured this window, no effect; the lever was here all along.
From the 2026-09-07 correlation measurement window (`CORRELATION_REVIEW_0907.md`, `corr_dump_9-7_0437am.json`). This — not scene detection (#5) — is the dominant over-split driver.
## Data (88 calls in a 5h, 2-node window)
`corr_consensus` × cheap-LLM `corr_llm_action` × resulting `corr_path`:
| consensus | cheap LLM said | count | → `new` |
|---|---|---|---|
| agreed | — | 27 | 20 |
| rules_only | — | 19 | 3 |
| **tiebreak** | **link** | 10 | **8** |
| tiebreak | new | 5 | 0 |
| **tiebreak** | **orphan** | 25 | **24** |
- `tiebreak` ran on **40 / 88 calls (45%)** — nearly every call hits the expensive smart model. Large Gemini spend on its own.
- Cheap LLM wanted **link** → tiebreaker chose "new" **8/10**.
- Cheap LLM wanted **orphan** → tiebreaker opened a **new incident 24/25**.
- **32 of 55 window "new" incidents** came out of a tiebreak where the cheap LLM wanted to link or orphan.
The tiebreaker's own `_RULES` (shared in `llm_correlator.py`) say *"orphan when in doubt — conservative is always correct"* and *"new only if the call has a clear incident_type AND describes a distinct, identifiable scene."* Its behaviour is the opposite.
## Concrete cases
- `f53119a5` — same TG-383 event ("female backed up the ramp and left", 05:12–05:16) landed in **two** incidents (`daa58459` "Vehicle Leaving Scene", `e55de831` "Vehicle Check at ramp from Midland"). `corr_consensus: tiebreak`, `corr_llm_action: link` → `corr_path: new` + double-linked.
- Mohegan Park Ave (TG 9560): `4e4a7864` "Nuisance Alarm" (07:54:57) + `0a95dc40` "Noise Complaint" (07:55:52) — same car-alarms call 55s apart. 2nd call `corr_llm_action: link` → `corr_path: new`.
## Where to look
- `llm_correlator.tiebreak()` + `_build_tiebreak_prompt()` — is the prompt biased toward "new"? Does it actually instruct the model that resolving a link/new disagreement toward *link* is the conservative choice for an over-splitting system?
- `_inc_summary()` — the active-incident block the tiebreaker sees is `id | type | loc | units[:6] | tags[:4] | idle`. No transcript, no call list. That may be too thin for the model to justify a link, so it defaults to "new".
- `decisions_agree()` / the consensus gate in `upload.py:_correlate_with_consensus` — 40/88 going to tiebreak suggests the cheap tier and rules disagree far more than expected; check whether the cheap LLM is being fed enough to agree with an obvious link.
- Interaction with #96: `corr_debug` is per-call last-write-wins, so for multi-scene calls these fields are a splice. The single-scene majority makes the aggregate directionally sound but a per-scene record (#96) is needed before tuning against exact numbers.
## Related
- Geocode coverage is near-zero (`location_proximity` fired 5× in 154 calls), so the location-fit path can't back up a link — "Ave" vs "Avenue" splits. Worth a normalisation pass (St/Street, Ave/Avenue, Rd/Road) either as its own issue or folded here.
- #103 (unit-overlap veto dark on `None` embedding), #86 (0.82 veto literal) — same bounding problem from the rules side.
- #5 (scene over-split prompt) — measured this window, no effect; the lever was here all along.
The reframed _RULES text changed nothing the tiebreaker outputs.
The call-by-call read reframes the bug
It is not "one conversation shattered into 5–10 incidents." Reading transcripts, the 1-call incidents are almost all different transmissions, and ~35–40 of the 80 window incidents are non-events that should never have been incidents: unit check-ins ("114 Paul, Metro Central. 10-4"), clearings ("6-8 show me 98"), roll call ("stand by for roll call"), call-number admin. The genuinely distinct events (property retrieval, subject check, Pelham Station, trespassing MP 13.7, MVA, fall injury) are correctly separate.
Mechanism, every time: corr_llm_action=orphan · corr_rules_action=new · corr_consensus=tiebreak → new. The cheap LLM reads the transcript and correctly says "not an incident"; rules says new only because there's no link candidate; the tiebreaker sides with rules ~21/21.
Recommendation — stop iterating the prompt, change the consensus logic
When corr_llm_action=="orphan" and corr_rules_action=="new" (rules has only "no candidate", not a positive event signal) → resolve to orphan, do not escalate. The LLM read the transcript; the rules engine didn't. Removes ~21/80 of this window's incidents. Small change in _correlate_with_consensus, no prompt. Guard: still escalate when rules has a real positive (unit-continuity, location+unit-overlap, fast/disambig).
Tighten corr_path=location — 3–4 clear over-merges this window (8 Hill St + Ossining restroom; two different churches an hour apart; a tg383 harassment + a tg9048 call). Needs unit overlap OR tight geocoded proximity, not a shared road token. Ties to #86 / #103.
Re-measure whether the 43% tiebreak tier earns its cost once (1) lands (#35, #96).
## Measurement window #2 (code bccb3e0 = #116) — result: no measurable improvement
Dump `corr_dump_9-7_pm.json`, ~4h, 2 nodes. Full call-by-call write-up in `Version 5C/CORRELATION_REVIEW_0907b.md`.
| metric | window #1 (pre-#116) | window #2 (#116) |
|---|---|---|
| new-incident rate | 62% | 57% (99/173) |
| tiebreak escalation | 45% | 43% (75/173) |
| single-call incidents | ~50% | 62% (62/100) |
| `llm=orphan` overridden to `new` by rules→tiebreak | 24/25 | ~21/21 |
The reframed `_RULES` text changed nothing the tiebreaker outputs.
### The call-by-call read reframes the bug
It is **not** "one conversation shattered into 5–10 incidents." Reading transcripts, the 1-call incidents are almost all *different* transmissions, and **~35–40 of the 80 window incidents are non-events that should never have been incidents**: unit check-ins ("114 Paul, Metro Central. 10-4"), clearings ("6-8 show me 98"), roll call ("stand by for roll call"), call-number admin. The genuinely distinct events (property retrieval, subject check, Pelham Station, trespassing MP 13.7, MVA, fall injury) are *correctly* separate.
Mechanism, every time: `corr_llm_action=orphan` · `corr_rules_action=new` · `corr_consensus=tiebreak` → **new**. The cheap LLM reads the transcript and correctly says "not an incident"; rules says `new` only because there's no link candidate; the tiebreaker sides with rules ~21/21.
### Recommendation — stop iterating the prompt, change the consensus logic
1. **When `corr_llm_action=="orphan"` and `corr_rules_action=="new"` (rules has only "no candidate", not a positive event signal) → resolve to `orphan`, do not escalate.** The LLM read the transcript; the rules engine didn't. Removes ~21/80 of this window's incidents. Small change in `_correlate_with_consensus`, no prompt. Guard: still escalate when rules has a real positive (`unit-continuity`, `location`+unit-overlap, `fast/disambig`).
2. **Tighten `corr_path=location`** — 3–4 clear over-merges this window (8 Hill St + Ossining restroom; two different churches an hour apart; a tg383 harassment + a tg9048 call). Needs unit overlap OR tight geocoded proximity, not a shared road token. Ties to #86 / #103.
3. Re-measure whether the 43% tiebreak tier earns its cost once (1) lands (#35, #96).
The consensus gate is implemented in PR #125 (branch fix/115-consensus-orphan-gate), pending review.
Fix 1 (primary)_correlate_with_consensus in routers/upload.py: when the cheap LLM says orphan and rules says new with no positive event signal, resolve to orphan and skip the tiebreak call. Guard: llm==orphan and rules==new and not _rules_has_positive_event_signal(rules_decision), where a positive signal is corr_fit_signal in {unit_overlap, location_proximity} or corr_path in {unit-continuity, location, fast/disambig, fast/single}. A bare new (empty corr_debug / fit signal in {None, none, thin_recency}) is gated; a real signal still escalates to the tiebreaker. The gated outcome records corr_consensus="llm_orphan_gate" and keeps corr_llm_reasoning / corr_rules_action / corr_llm_action.
Fix 2 (secondary)corr_path=location in incident_correlator.py: a sub-0.5km distance alone no longer links. Now requires unit overlap with the candidate OR distance under _LOCATION_TIGHT_PROXIMITY_KM = 0.2km; pursuit incidents keep their wide movement-validated radius. A surviving location link now also stamps corr_fit_signal. Proximity half done for real (no TODO).
Tests: new tests/test_consensus_gate.py (13 cases, tiebreak spied). Full c2-core suite 309 -> 322 passing, green.
The consensus gate is implemented in **PR #125** (branch `fix/115-consensus-orphan-gate`), pending review.
**Fix 1 (primary)** `_correlate_with_consensus` in `routers/upload.py`: when the cheap LLM says `orphan` and rules says `new` with no positive event signal, resolve to `orphan` and skip the `tiebreak` call. Guard:
`llm==orphan and rules==new and not _rules_has_positive_event_signal(rules_decision)`, where a positive signal is `corr_fit_signal in {unit_overlap, location_proximity}` or `corr_path in {unit-continuity, location, fast/disambig, fast/single}`. A bare `new` (empty corr_debug / fit signal in {None, none, thin_recency}) is gated; a real signal still escalates to the tiebreaker. The gated outcome records `corr_consensus="llm_orphan_gate"` and keeps `corr_llm_reasoning` / `corr_rules_action` / `corr_llm_action`.
**Fix 2 (secondary)** `corr_path=location` in `incident_correlator.py`: a sub-0.5km distance alone no longer links. Now requires unit overlap with the candidate OR distance under `_LOCATION_TIGHT_PROXIMITY_KM = 0.2km`; pursuit incidents keep their wide movement-validated radius. A surviving `location` link now also stamps `corr_fit_signal`. Proximity half done for real (no TODO).
Tests: new `tests/test_consensus_gate.py` (13 cases, tiebreak spied). Full c2-core suite 309 -> 322 passing, green.
MUST-FIX 1 — dead safety net. Confirmed: _rules_has_positive_event_signal read rules_decision["corr_debug"], which is empty at preview time for action=="new" (corr_path/fit-signal written at apply time). The gate was firing on every llm=orphan/rules=new disagreement. Replaced with _call_is_substanceless(ctx) — gates to orphan ONLY when all of: severity not moderate/major, no ctx["call_vehicles"|"coords"|"tags"], and no incident in ctx["recent"] on the same system+talkgroup. Otherwise escalates to tiebreak as before. The vehicle/geocode/tag test is incident_correlator.has_event_substance, factored out of the creation gate (~line 1372) and shared so the two can't drift.
MUST-FIX 2 — permanent gating. Decision: longer link-only sweep budget for gated orphans (GATED_ORPHAN_SWEEP_ATTEMPTS=10 vs 3), NOT creation-eligibility. Making the sweep create_if_new=True for these would be inert — the rules creation gate resolves no incident_type for a substanceless call and returns orphan regardless — and minting an incident from a content-free call is the bug that gate prevents. The cascade is broken instead by the new same-talkgroup check: once any substantive call opens the incident, later thin calls on that TG escalate rather than gate.
Fix-2 nits. Location path now scores every in-radius candidate and links the nearest qualifying one (was break on first in unsorted recent). Unit-overlap location links now write corr_fit_signal="location_unit_overlap" so they stop merging into the fast-path unit_overlap bucket in the admin.py histogram (#35).
Tests. test_consensus_gate.py 13 -> 16 cases; corr_debug-signal cases replaced with ctx-substance cases (severity/coords/tags/vehicles/same-tg); nearest-wins location test added; the location guard tests now assert they reach the new guard (caplog on the skip line, plus the distinct fit_signal). Full drb-c2-core suite 322 -> 325, green (sandboxed Linux pytest).
PR #125 revised after review (commit dd42657).
**MUST-FIX 1 — dead safety net.** Confirmed: `_rules_has_positive_event_signal` read `rules_decision["corr_debug"]`, which is empty at preview time for `action=="new"` (corr_path/fit-signal written at apply time). The gate was firing on every llm=orphan/rules=new disagreement. Replaced with `_call_is_substanceless(ctx)` — gates to orphan ONLY when all of: severity not moderate/major, no `ctx["call_vehicles"|"coords"|"tags"]`, and no incident in `ctx["recent"]` on the same system+talkgroup. Otherwise escalates to tiebreak as before. The vehicle/geocode/tag test is `incident_correlator.has_event_substance`, factored out of the creation gate (~line 1372) and shared so the two can't drift.
**MUST-FIX 2 — permanent gating.** Decision: longer link-only sweep budget for gated orphans (`GATED_ORPHAN_SWEEP_ATTEMPTS=10` vs 3), NOT creation-eligibility. Making the sweep `create_if_new=True` for these would be inert — the rules creation gate resolves no incident_type for a substanceless call and returns orphan regardless — and minting an incident from a content-free call is the bug that gate prevents. The cascade is broken instead by the new same-talkgroup check: once any substantive call opens the incident, later thin calls on that TG escalate rather than gate.
**Fix-2 nits.** Location path now scores every in-radius candidate and links the nearest qualifying one (was `break` on first in unsorted `recent`). Unit-overlap location links now write `corr_fit_signal="location_unit_overlap"` so they stop merging into the fast-path `unit_overlap` bucket in the admin.py histogram (#35).
**Tests.** test_consensus_gate.py 13 -> 16 cases; corr_debug-signal cases replaced with ctx-substance cases (severity/coords/tags/vehicles/same-tg); nearest-wins location test added; the location guard tests now assert they reach the new guard (caplog on the skip line, plus the distinct fit_signal). Full drb-c2-core suite 322 -> 325, green (sandboxed Linux pytest).
Two drb-correlation-review passes, both must-fixes applied:
Fix 1 — the gate.llm=orphan + rules=new resolves to orphan without a tiebreak only when _call_is_substanceless(ctx):
severity not moderate/major
no coords, tags, or vehicles (incident_correlator.has_event_substance, extracted from the creation gate so the two can't drift)
no recent incident on the same system+talkgroup (the "unit dispatched, thin ack 10-30s later" case — keeps the tiebreak)
no resolved incident_type and not reassignment (round-2 fix: a routine-severity fire/medical call, or a unit pulled to a new job, would otherwise be dropped where rules would open an incident)
Any one false → unchanged escalation to tiebreak.
Fix 2 — corr_path=location. A non-pursuit location link now needs unit overlap OR ≤ 0.2 km (was: anywhere in the 0.5 km radius). Unit-overlap links write corr_fit_signal="location_unit_overlap" (distinct bucket for the #35 tally); distance-only keep location_proximity. Candidates are scored and the nearest qualifying one is linked (was: break on first in an unsorted list).
Permanence. Gated orphans get GATED_ORPHAN_SWEEP_ATTEMPTS=10 (vs 3) in the re-correlation sweep, link-only — so a substantive call landing on the TG within ~20 min still pulls the earlier thin calls in. Not made creation-eligible (a substanceless call resolves no incident_type, so it'd be inert, and minting an incident from a content-free call is the failure this gate prevents).
Residual (not blocking, follow-ups):
_LOCATION_TIGHT_PROXIMITY_KM = 0.2 is a hardcoded literal beside the configurable location_proximity_km — same shape as #86. Should move to config.
On a busy dispatch backbone a recent same-TG incident is nearly always live, so the gate mostly fires on quiet channels — recovers less of the window-#2 21/21 than the raw count suggests. Measure the llm_orphan_gate bucket in a window #3 (it appears in orphaned_calls, not the linked histogram).
The upstream fix (dispatch-vs-chatter classifier so "10-4"/"98"/roll-call never create a scene) is still the real lever — this PR is the stopgap.
## PR #125 — ready for merge (commit 15a9d10, 3 commits, sandboxed pytest 327 green)
Two `drb-correlation-review` passes, both must-fixes applied:
**Fix 1 — the gate.** `llm=orphan` + `rules=new` resolves to **orphan without a tiebreak** only when `_call_is_substanceless(ctx)`:
- severity not moderate/major
- no coords, tags, or vehicles (`incident_correlator.has_event_substance`, extracted from the creation gate so the two can't drift)
- no recent incident on the same system+talkgroup (the "unit dispatched, thin ack 10-30s later" case — keeps the tiebreak)
- **no resolved `incident_type` and not `reassignment`** (round-2 fix: a routine-severity fire/medical call, or a unit pulled to a new job, would otherwise be dropped where rules would open an incident)
Any one false → unchanged escalation to `tiebreak`.
**Fix 2 — `corr_path=location`.** A non-pursuit location link now needs unit overlap OR ≤ 0.2 km (was: anywhere in the 0.5 km radius). Unit-overlap links write `corr_fit_signal="location_unit_overlap"` (distinct bucket for the #35 tally); distance-only keep `location_proximity`. Candidates are scored and the nearest qualifying one is linked (was: `break` on first in an unsorted list).
**Permanence.** Gated orphans get `GATED_ORPHAN_SWEEP_ATTEMPTS=10` (vs 3) in the re-correlation sweep, link-only — so a substantive call landing on the TG within ~20 min still pulls the earlier thin calls in. Not made creation-eligible (a substanceless call resolves no `incident_type`, so it'd be inert, and minting an incident from a content-free call is the failure this gate prevents).
**Residual (not blocking, follow-ups):**
- `_LOCATION_TIGHT_PROXIMITY_KM = 0.2` is a hardcoded literal beside the configurable `location_proximity_km` — same shape as #86. Should move to config.
- On a busy dispatch backbone a recent same-TG incident is nearly always live, so the gate mostly fires on quiet channels — recovers less of the window-#2 21/21 than the raw count suggests. **Measure the `llm_orphan_gate` bucket in a window #3** (it appears in `orphaned_calls`, not the linked histogram).
- The upstream fix (dispatch-vs-chatter classifier so "10-4"/"98"/roll-call never create a scene) is still the real lever — this PR is the stopgap.
corr_consensus=="llm_orphan_gate" count: 0 in the entire dump
24 linked calls this window have the exact rules=new, llm=orphan shape #125 targets — all 24 still escalated to tiebreak → new, same as window #2's 21/21
Root cause (confirmed 22/24): the gate's "skip if a recent incident exists on the same talkgroup" escape hatch uses a 2-hour lookback. The three busiest channels produce 3-13 other incidents per 2h, so that escape hatch is satisfied almost unconditionally — it disables the gate on exactly the channels it exists for. 2/24 have zero explainable recent-same-tg candidates and still weren't gated — needs an engineer to read the actual ctx["recent"] construction, not inferred from the dump.
Decision: don't ship a third consensus-layer tweak. Shrink the escape-hatch window from 2h to a few minutes (matching what "thin ack right after dispatch" actually means — fast/thin already handles true immediate follow-ups upstream of this branch), investigate the 2 unexplained misses, then re-measure. The durable fix is still the unscoped upstream chatter classifier — two iterations at the consensus layer (#116, #125) have now both measured as no-ops on the top-line rate.
## Window #3 result (code 07ff9ba193 = #125): the gate fired ZERO times
100 incidents, 91 orphans, 4h live. Full write-up `Version 5C/CORRELATION_REVIEW_0912.md`.
- new-rate unchanged: 57% (was 57% in window #2)
- `corr_consensus=="llm_orphan_gate"` count: **0** in the entire dump
- 24 linked calls this window have the exact `rules=new, llm=orphan` shape #125 targets — all 24 still escalated to `tiebreak` → `new`, same as window #2's 21/21
**Root cause (confirmed 22/24):** the gate's "skip if a recent incident exists on the same talkgroup" escape hatch uses a 2-hour lookback. The three busiest channels produce 3-13 other incidents per 2h, so that escape hatch is satisfied almost unconditionally — it disables the gate on exactly the channels it exists for. 2/24 have zero explainable recent-same-tg candidates and still weren't gated — needs an engineer to read the actual `ctx["recent"]` construction, not inferred from the dump.
**Decision:** don't ship a third consensus-layer tweak. Shrink the escape-hatch window from 2h to a few minutes (matching what "thin ack right after dispatch" actually means — `fast/thin` already handles true immediate follow-ups upstream of this branch), investigate the 2 unexplained misses, then re-measure. The durable fix is still the unscoped upstream chatter classifier — two iterations at the consensus layer (#116, #125) have now both measured as no-ops on the top-line rate.
Opened PR #126: shrinks _recent_incident_on_same_talkgroup's lookback from 2h (correlation_window_hours) to settings.tg_dispatch_thin_idle_minutes (5 min) — the same recency bound the fast/thin path already uses for "dispatch, thin ack" — which is what the 2h version was letting through unconditionally on busy channels (window #3 measured 0/24 gate fires against the exact target shape).
Re: the 2 unexplained misses — confirmed ctx["recent"] is status=="active" incidents with over-capacity ones dropped (_build_context/_drop_capped), not a full collection scan, so a resolved/capped same-tg incident is invisible to the check even though it's chronologically recent. That's a real, documented gap (docstring + TODO(server-26#115) left in place), but it doesn't explain the review's specific 2 misses — those already had zero same-tg candidates by a full-collection scan, active or not, so some other _call_is_substanceless condition must have been true for those two. Not reproducible without the live dump, so left as-is rather than guessed at.
Sandboxed pytest: 327 -> 329 passed, including a regression test that fails on main and passes on the branch.
Opened PR #126: shrinks `_recent_incident_on_same_talkgroup`'s lookback from 2h (`correlation_window_hours`) to `settings.tg_dispatch_thin_idle_minutes` (5 min) — the same recency bound the fast/thin path already uses for "dispatch, thin ack" — which is what the 2h version was letting through unconditionally on busy channels (window #3 measured 0/24 gate fires against the exact target shape).
Re: the 2 unexplained misses — confirmed `ctx["recent"]` is `status=="active"` incidents with over-capacity ones dropped (`_build_context`/`_drop_capped`), not a full collection scan, so a resolved/capped same-tg incident is invisible to the check even though it's chronologically recent. That's a real, documented gap (docstring + `TODO(server-26#115)` left in place), but it doesn't explain the review's specific 2 misses — those already had zero same-tg candidates by a full-collection scan, active or not, so some other `_call_is_substanceless` condition must have been true for those two. Not reproducible without the live dump, so left as-is rather than guessed at.
Sandboxed pytest: 327 -> 329 passed, including a regression test that fails on `main` and passes on the branch.
PR #126 ready — fixes why #125 fired 0/24 times in window #3. Root cause: the "recent same-talkgroup incident" escape hatch used a 2h lookback, satisfied almost unconditionally on busy dispatch channels. Now 5 min (dispatch) / 15 min (tactical), mirroring the fast/thin path's own bound-selection. Added corr_gate_veto to corr_debug + the admin endpoint so window #4 measures the escape rate directly instead of reconstructing it from the raw dump. 2 review rounds, 332 tests green. Details on PR #126.
Expectation: the hatch still lacks a single-candidate requirement (the fast/thin path it mirrors has one), so expect partial recovery of the 24 target calls next window, not all of them — that's the next tuning decision, not a defect in this PR.
PR #126 ready — fixes why #125 fired 0/24 times in window #3. Root cause: the "recent same-talkgroup incident" escape hatch used a 2h lookback, satisfied almost unconditionally on busy dispatch channels. Now 5 min (dispatch) / 15 min (tactical), mirroring the fast/thin path's own bound-selection. Added `corr_gate_veto` to corr_debug + the admin endpoint so window #4 measures the escape rate directly instead of reconstructing it from the raw dump. 2 review rounds, 332 tests green. Details on PR #126.
Expectation: the hatch still lacks a single-candidate requirement (the fast/thin path it mirrors has one), so expect partial recovery of the 24 target calls next window, not all of them — that's the next tuning decision, not a defect in this PR.
Window #4 result — 0/22, but #126 was never actually tested.
Full writeup: CORRELATION_REVIEW_0913.md.
The debug panel's "Incidents (limit)=100" is NOT time-boxed — 79/287 (28%) of the raw dump's scene-decisions were from 2026-09-12 or earlier, contaminating the first pass. After filtering to calls actually started inside the window (16:34-20:26 UTC), the true orphan-vs-new population is 22 calls, 0 gate fires.
0/22 hit the escape hatch #126 fixed. The 9 unexplained gate_veto=None records in the raw dump were all out-of-window contamination, not evidence against #126. This window gives no signal on whether #126 works — real traffic never exercised it.
18/22 (82%) vetoed on "type" — and every one of those 18 transcripts is visibly pure radio housekeeping (roll calls, 10-4 acks, plate readbacks) with the call's own final incident_type field == None. ctx["incident_type"] (read at correlation time, in-memory from the scene's own extraction) was truthy anyway. Either extraction is over-typing trivial chatter, or a sibling scene's type is leaking into this scene's ctx before its own extraction overwrites the shared top-level field — can't tell apart from this dump because #96/#114's per-scene scenes map doesn't capture incident_type per scene.
This is the actual reason the gate hasn't fired in either window — not the escape hatch. Fragmentation shape is unchanged: 46/62 (74%) of incidents started in-window are single-call, same as every prior window.
Per CORRELATION_REVIEW_0912.md's ceiling math this doesn't reopen#115 — filing two follow-ups instead (linked below) and standing by the existing call to deprioritize further gate tuning in favor of #127.
Closing scope on this issue's measurement question: #126 is unverified-but-not-refuted, and the real live blocker is a different, newly-identified mechanism.
**Window #4 result — 0/22, but #126 was never actually tested.**
Full writeup: `CORRELATION_REVIEW_0913.md`.
The debug panel's "Incidents (limit)=100" is NOT time-boxed — 79/287 (28%) of the raw dump's scene-decisions were from 2026-09-12 or earlier, contaminating the first pass. After filtering to calls actually started inside the window (16:34-20:26 UTC), the true orphan-vs-new population is **22 calls, 0 gate fires**.
**0/22 hit the escape hatch #126 fixed.** The 9 unexplained gate_veto=None records in the raw dump were all out-of-window contamination, not evidence against #126. This window gives no signal on whether #126 works — real traffic never exercised it.
**18/22 (82%) vetoed on `"type"`** — and every one of those 18 transcripts is visibly pure radio housekeeping (roll calls, 10-4 acks, plate readbacks) with the call's own final `incident_type` field == None. `ctx["incident_type"]` (read at correlation time, in-memory from the scene's own extraction) was truthy anyway. Either extraction is over-typing trivial chatter, or a sibling scene's type is leaking into this scene's ctx before its own extraction overwrites the shared top-level field — can't tell apart from this dump because #96/#114's per-scene `scenes` map doesn't capture `incident_type` per scene.
**This is the actual reason the gate hasn't fired in either window — not the escape hatch.** Fragmentation shape is unchanged: 46/62 (74%) of incidents started in-window are single-call, same as every prior window.
Per CORRELATION_REVIEW_0912.md's ceiling math this doesn't reopen #115 — filing two follow-ups instead (linked below) and standing by the existing call to deprioritize further gate tuning in favor of #127.
Closing scope on this issue's measurement question: #126 is unverified-but-not-refuted, and the real live blocker is a different, newly-identified mechanism.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
From the 2026-09-07 correlation measurement window (
CORRELATION_REVIEW_0907.md,corr_dump_9-7_0437am.json). This — not scene detection (#5) — is the dominant over-split driver.Data (88 calls in a 5h, 2-node window)
corr_consensus× cheap-LLMcorr_llm_action× resultingcorr_path:newtiebreakran on 40 / 88 calls (45%) — nearly every call hits the expensive smart model. Large Gemini spend on its own.The tiebreaker's own
_RULES(shared inllm_correlator.py) say "orphan when in doubt — conservative is always correct" and "new only if the call has a clear incident_type AND describes a distinct, identifiable scene." Its behaviour is the opposite.Concrete cases
f53119a5— same TG-383 event ("female backed up the ramp and left", 05:12–05:16) landed in two incidents (daa58459"Vehicle Leaving Scene",e55de831"Vehicle Check at ramp from Midland").corr_consensus: tiebreak,corr_llm_action: link→corr_path: new+ double-linked.4e4a7864"Nuisance Alarm" (07:54:57) +0a95dc40"Noise Complaint" (07:55:52) — same car-alarms call 55s apart. 2nd callcorr_llm_action: link→corr_path: new.Where to look
llm_correlator.tiebreak()+_build_tiebreak_prompt()— is the prompt biased toward "new"? Does it actually instruct the model that resolving a link/new disagreement toward link is the conservative choice for an over-splitting system?_inc_summary()— the active-incident block the tiebreaker sees isid | type | loc | units[:6] | tags[:4] | idle. No transcript, no call list. That may be too thin for the model to justify a link, so it defaults to "new".decisions_agree()/ the consensus gate inupload.py:_correlate_with_consensus— 40/88 going to tiebreak suggests the cheap tier and rules disagree far more than expected; check whether the cheap LLM is being fed enough to agree with an obvious link.corr_debugis per-call last-write-wins, so for multi-scene calls these fields are a splice. The single-scene majority makes the aggregate directionally sound but a per-scene record (#96) is needed before tuning against exact numbers.Related
location_proximityfired 5× in 154 calls), so the location-fit path can't back up a link — "Ave" vs "Avenue" splits. Worth a normalisation pass (St/Street, Ave/Avenue, Rd/Road) either as its own issue or folded here.Noneembedding), #86 (0.82 veto literal) — same bounding problem from the rules side.Measurement window #2 (code
bccb3e0= #116) — result: no measurable improvementDump
corr_dump_9-7_pm.json, ~4h, 2 nodes. Full call-by-call write-up inVersion 5C/CORRELATION_REVIEW_0907b.md.llm=orphanoverridden tonewby rules→tiebreakThe reframed
_RULEStext changed nothing the tiebreaker outputs.The call-by-call read reframes the bug
It is not "one conversation shattered into 5–10 incidents." Reading transcripts, the 1-call incidents are almost all different transmissions, and ~35–40 of the 80 window incidents are non-events that should never have been incidents: unit check-ins ("114 Paul, Metro Central. 10-4"), clearings ("6-8 show me 98"), roll call ("stand by for roll call"), call-number admin. The genuinely distinct events (property retrieval, subject check, Pelham Station, trespassing MP 13.7, MVA, fall injury) are correctly separate.
Mechanism, every time:
corr_llm_action=orphan·corr_rules_action=new·corr_consensus=tiebreak→ new. The cheap LLM reads the transcript and correctly says "not an incident"; rules saysnewonly because there's no link candidate; the tiebreaker sides with rules ~21/21.Recommendation — stop iterating the prompt, change the consensus logic
corr_llm_action=="orphan"andcorr_rules_action=="new"(rules has only "no candidate", not a positive event signal) → resolve toorphan, do not escalate. The LLM read the transcript; the rules engine didn't. Removes ~21/80 of this window's incidents. Small change in_correlate_with_consensus, no prompt. Guard: still escalate when rules has a real positive (unit-continuity,location+unit-overlap,fast/disambig).corr_path=location— 3–4 clear over-merges this window (8 Hill St + Ossining restroom; two different churches an hour apart; a tg383 harassment + a tg9048 call). Needs unit overlap OR tight geocoded proximity, not a shared road token. Ties to #86 / #103.The consensus gate is implemented in PR #125 (branch
fix/115-consensus-orphan-gate), pending review.Fix 1 (primary)
_correlate_with_consensusinrouters/upload.py: when the cheap LLM saysorphanand rules saysnewwith no positive event signal, resolve toorphanand skip thetiebreakcall. Guard:llm==orphan and rules==new and not _rules_has_positive_event_signal(rules_decision), where a positive signal iscorr_fit_signal in {unit_overlap, location_proximity}orcorr_path in {unit-continuity, location, fast/disambig, fast/single}. A barenew(empty corr_debug / fit signal in {None, none, thin_recency}) is gated; a real signal still escalates to the tiebreaker. The gated outcome recordscorr_consensus="llm_orphan_gate"and keepscorr_llm_reasoning/corr_rules_action/corr_llm_action.Fix 2 (secondary)
corr_path=locationinincident_correlator.py: a sub-0.5km distance alone no longer links. Now requires unit overlap with the candidate OR distance under_LOCATION_TIGHT_PROXIMITY_KM = 0.2km; pursuit incidents keep their wide movement-validated radius. A survivinglocationlink now also stampscorr_fit_signal. Proximity half done for real (no TODO).Tests: new
tests/test_consensus_gate.py(13 cases, tiebreak spied). Full c2-core suite 309 -> 322 passing, green.PR #125 revised after review (commit
dd42657).MUST-FIX 1 — dead safety net. Confirmed:
_rules_has_positive_event_signalreadrules_decision["corr_debug"], which is empty at preview time foraction=="new"(corr_path/fit-signal written at apply time). The gate was firing on every llm=orphan/rules=new disagreement. Replaced with_call_is_substanceless(ctx)— gates to orphan ONLY when all of: severity not moderate/major, noctx["call_vehicles"|"coords"|"tags"], and no incident inctx["recent"]on the same system+talkgroup. Otherwise escalates to tiebreak as before. The vehicle/geocode/tag test isincident_correlator.has_event_substance, factored out of the creation gate (~line 1372) and shared so the two can't drift.MUST-FIX 2 — permanent gating. Decision: longer link-only sweep budget for gated orphans (
GATED_ORPHAN_SWEEP_ATTEMPTS=10vs 3), NOT creation-eligibility. Making the sweepcreate_if_new=Truefor these would be inert — the rules creation gate resolves no incident_type for a substanceless call and returns orphan regardless — and minting an incident from a content-free call is the bug that gate prevents. The cascade is broken instead by the new same-talkgroup check: once any substantive call opens the incident, later thin calls on that TG escalate rather than gate.Fix-2 nits. Location path now scores every in-radius candidate and links the nearest qualifying one (was
breakon first in unsortedrecent). Unit-overlap location links now writecorr_fit_signal="location_unit_overlap"so they stop merging into the fast-pathunit_overlapbucket in the admin.py histogram (#35).Tests. test_consensus_gate.py 13 -> 16 cases; corr_debug-signal cases replaced with ctx-substance cases (severity/coords/tags/vehicles/same-tg); nearest-wins location test added; the location guard tests now assert they reach the new guard (caplog on the skip line, plus the distinct fit_signal). Full drb-c2-core suite 322 -> 325, green (sandboxed Linux pytest).
PR #125 — ready for merge (commit
15a9d10, 3 commits, sandboxed pytest 327 green)Two
drb-correlation-reviewpasses, both must-fixes applied:Fix 1 — the gate.
llm=orphan+rules=newresolves to orphan without a tiebreak only when_call_is_substanceless(ctx):incident_correlator.has_event_substance, extracted from the creation gate so the two can't drift)incident_typeand notreassignment(round-2 fix: a routine-severity fire/medical call, or a unit pulled to a new job, would otherwise be dropped where rules would open an incident)Any one false → unchanged escalation to
tiebreak.Fix 2 —
corr_path=location. A non-pursuit location link now needs unit overlap OR ≤ 0.2 km (was: anywhere in the 0.5 km radius). Unit-overlap links writecorr_fit_signal="location_unit_overlap"(distinct bucket for the #35 tally); distance-only keeplocation_proximity. Candidates are scored and the nearest qualifying one is linked (was:breakon first in an unsorted list).Permanence. Gated orphans get
GATED_ORPHAN_SWEEP_ATTEMPTS=10(vs 3) in the re-correlation sweep, link-only — so a substantive call landing on the TG within ~20 min still pulls the earlier thin calls in. Not made creation-eligible (a substanceless call resolves noincident_type, so it'd be inert, and minting an incident from a content-free call is the failure this gate prevents).Residual (not blocking, follow-ups):
_LOCATION_TIGHT_PROXIMITY_KM = 0.2is a hardcoded literal beside the configurablelocation_proximity_km— same shape as #86. Should move to config.llm_orphan_gatebucket in a window #3 (it appears inorphaned_calls, not the linked histogram).Window #3 result (code
07ff9ba193= #125): the gate fired ZERO times100 incidents, 91 orphans, 4h live. Full write-up
Version 5C/CORRELATION_REVIEW_0912.md.corr_consensus=="llm_orphan_gate"count: 0 in the entire dumprules=new, llm=orphanshape #125 targets — all 24 still escalated totiebreak→new, same as window #2's 21/21Root cause (confirmed 22/24): the gate's "skip if a recent incident exists on the same talkgroup" escape hatch uses a 2-hour lookback. The three busiest channels produce 3-13 other incidents per 2h, so that escape hatch is satisfied almost unconditionally — it disables the gate on exactly the channels it exists for. 2/24 have zero explainable recent-same-tg candidates and still weren't gated — needs an engineer to read the actual
ctx["recent"]construction, not inferred from the dump.Decision: don't ship a third consensus-layer tweak. Shrink the escape-hatch window from 2h to a few minutes (matching what "thin ack right after dispatch" actually means —
fast/thinalready handles true immediate follow-ups upstream of this branch), investigate the 2 unexplained misses, then re-measure. The durable fix is still the unscoped upstream chatter classifier — two iterations at the consensus layer (#116, #125) have now both measured as no-ops on the top-line rate.Opened PR #126: shrinks
_recent_incident_on_same_talkgroup's lookback from 2h (correlation_window_hours) tosettings.tg_dispatch_thin_idle_minutes(5 min) — the same recency bound the fast/thin path already uses for "dispatch, thin ack" — which is what the 2h version was letting through unconditionally on busy channels (window #3 measured 0/24 gate fires against the exact target shape).Re: the 2 unexplained misses — confirmed
ctx["recent"]isstatus=="active"incidents with over-capacity ones dropped (_build_context/_drop_capped), not a full collection scan, so a resolved/capped same-tg incident is invisible to the check even though it's chronologically recent. That's a real, documented gap (docstring +TODO(server-26#115)left in place), but it doesn't explain the review's specific 2 misses — those already had zero same-tg candidates by a full-collection scan, active or not, so some other_call_is_substancelesscondition must have been true for those two. Not reproducible without the live dump, so left as-is rather than guessed at.Sandboxed pytest: 327 -> 329 passed, including a regression test that fails on
mainand passes on the branch.PR #126 ready — fixes why #125 fired 0/24 times in window #3. Root cause: the "recent same-talkgroup incident" escape hatch used a 2h lookback, satisfied almost unconditionally on busy dispatch channels. Now 5 min (dispatch) / 15 min (tactical), mirroring the fast/thin path's own bound-selection. Added
corr_gate_vetoto corr_debug + the admin endpoint so window #4 measures the escape rate directly instead of reconstructing it from the raw dump. 2 review rounds, 332 tests green. Details on PR #126.Expectation: the hatch still lacks a single-candidate requirement (the fast/thin path it mirrors has one), so expect partial recovery of the 24 target calls next window, not all of them — that's the next tuning decision, not a defect in this PR.
Window #4 result — 0/22, but #126 was never actually tested.
Full writeup:
CORRELATION_REVIEW_0913.md.The debug panel's "Incidents (limit)=100" is NOT time-boxed — 79/287 (28%) of the raw dump's scene-decisions were from 2026-09-12 or earlier, contaminating the first pass. After filtering to calls actually started inside the window (16:34-20:26 UTC), the true orphan-vs-new population is 22 calls, 0 gate fires.
0/22 hit the escape hatch #126 fixed. The 9 unexplained gate_veto=None records in the raw dump were all out-of-window contamination, not evidence against #126. This window gives no signal on whether #126 works — real traffic never exercised it.
18/22 (82%) vetoed on
"type"— and every one of those 18 transcripts is visibly pure radio housekeeping (roll calls, 10-4 acks, plate readbacks) with the call's own finalincident_typefield == None.ctx["incident_type"](read at correlation time, in-memory from the scene's own extraction) was truthy anyway. Either extraction is over-typing trivial chatter, or a sibling scene's type is leaking into this scene's ctx before its own extraction overwrites the shared top-level field — can't tell apart from this dump because #96/#114's per-scenescenesmap doesn't captureincident_typeper scene.This is the actual reason the gate hasn't fired in either window — not the escape hatch. Fragmentation shape is unchanged: 46/62 (74%) of incidents started in-window are single-call, same as every prior window.
Per CORRELATION_REVIEW_0912.md's ceiling math this doesn't reopen #115 — filing two follow-ups instead (linked below) and standing by the existing call to deprioritize further gate tuning in favor of #127.
Closing scope on this issue's measurement question: #126 is unverified-but-not-refuted, and the real live blocker is a different, newly-identified mechanism.