Per #115's decision (comments 2026-09-08 and 2026-09-12) and both CORRELATION_REVIEW_0907b.md / CORRELATION_REVIEW_0912.md: two consensus-layer fixes (#125, #126) have now both measured as no-ops on the top-line new-incident rate. The disagreement shape they target (corr_llm_action==orphan, corr_rules_action==new) is real but structurally small — backtesting a perfect version of that gate against window #3's own data caps out around 14/57 single-call "junk" incidents in a 4h window. 43/57 are outside the gate's scope entirely, and the largest single bucket (15/57) is agreed — both correlator tiers independently called it a new incident, with no disagreement for any consensus-layer fix to arbitrate.
Decision (both review docs): stop iterating the consensus layer. Move the fix upstream — a dispatch-vs-chatter classifier in scene extraction (intelligence.py) so radio housekeeping never produces a scene at all.
What this issue ships
drb-c2-core/app/internal/chatter_classifier.py — a pure function classify_chatter(transcript) -> (bool, reason) that recognizes two shapes, drawn from hand-labeled examples in the review docs cross-referenced against the real dumps (not invented regexes):
roll_call — transcript contains a roll-call announcement or line-up ("stand by for roll call", "Baker 1? Baker 1. Henry 1? ...")
bare_acknowledgement — every token in the transcript reduces to a callsign/10-code/unit-ID (any digit-led token) or a small fixed set of procedural filler words ("central", "affirmative", "10-4", "stand by", "show me", "clear", ...). Any other word — a location, an action, a description — leaves the transcript unclassified.
Deliberately conservative: when a transcript doesn't cleanly reduce, it returns (False, None) and the existing pipeline runs exactly as today. Precision over recall — a false positive here would (once live) silently mean a real event never gets a scene.
SHIPPED IN SHADOW MODE ONLY.intelligence.extract_scenes now computes the verdict alongside the existing _is_garbage_transcript / transcript_too_short gates and writes chatter_classifier_verdict (bool) and chatter_classifier_reason (str|None) onto the call doc — but does not skip scene extraction. Scene extraction runs exactly as it does today regardless of the verdict. The two fields are now surfaced in /admin's correlation-debug _call_summary, same pattern as corr_gate_veto (#115/#126), so the next live measurement window can report the real-world false-positive rate before anything is wired to actually skip extraction.
Backtest (offline, against the three existing dumps — see PR for script)
* false positive = a flagged call carrying tags, coords, non-routine severity, or one of the specific transcripts the review docs named as dangerous to drop (the major "extinguishing fire" call, the geocoded 911-hangup call, both pursuit updates, the "6 Alpha... Pelham Station" subject check, the property-retrieval call, the Maronex subject check, the milepost-13.7 trespass call) — all individually re-checked and confirmed (False, None).
Catch rate against a proxy chatter-universe (severity none/routine, no tags, no coords, no resolved incident_type — a superset of true housekeeping that also includes some mid-conversation follow-up fragments of real events, so this undercounts precision on the narrower true-chatter set): 55/343 (16%), 72/271 (27%), 27/201 (13%). Recall is intentionally low — the classifier only recognizes the two shapes above, not every housekeeping pattern (garbled spelled-out roll call like "Five units. Six hundred. Eighty-four. Ten-four.", ambiguous check-ins with a stray content word like "7-4, call Central. 6-9, location.", CID/plate/case-number readbacks that mix in other words) were deliberately left uncaught rather than risk a broader net.
Tests
drb-c2-core/tests/test_chatter_classifier.py — real transcripts pulled from the dumps and review docs, both directions: chatter examples that must classify True, and the review docs' own named-dangerous real events that must classify False.
Next step — NOT done here
This is shadow mode only. Nothing skips scene extraction yet. A # TODO(server-26#<this-issue>) at the call site marks exactly what has to be true before flipping it live: run at least one more live shadow-mode measurement window, confirm the false-positive rate against real production traffic is still zero (this backtest only proves it against three already-collected dumps), and only then wire the verdict to actually skip extract_scenes.
Per #115's decision (comments 2026-09-08 and 2026-09-12) and both `CORRELATION_REVIEW_0907b.md` / `CORRELATION_REVIEW_0912.md`: two consensus-layer fixes (#125, #126) have now both measured as no-ops on the top-line new-incident rate. The disagreement shape they target (`corr_llm_action==orphan`, `corr_rules_action==new`) is real but structurally small — backtesting a perfect version of that gate against window #3's own data caps out around 14/57 single-call "junk" incidents in a 4h window. 43/57 are outside the gate's scope entirely, and the largest single bucket (15/57) is `agreed` — both correlator tiers independently called it a new incident, with no disagreement for any consensus-layer fix to arbitrate.
**Decision (both review docs): stop iterating the consensus layer. Move the fix upstream — a dispatch-vs-chatter classifier in scene extraction (`intelligence.py`) so radio housekeeping never produces a scene at all.**
## What this issue ships
`drb-c2-core/app/internal/chatter_classifier.py` — a pure function `classify_chatter(transcript) -> (bool, reason)` that recognizes two shapes, drawn from hand-labeled examples in the review docs cross-referenced against the real dumps (not invented regexes):
- **`roll_call`** — transcript contains a roll-call announcement or line-up ("stand by for roll call", "Baker 1? Baker 1. Henry 1? ...")
- **`bare_acknowledgement`** — every token in the transcript reduces to a callsign/10-code/unit-ID (any digit-led token) or a small fixed set of procedural filler words ("central", "affirmative", "10-4", "stand by", "show me", "clear", ...). Any other word — a location, an action, a description — leaves the transcript unclassified.
Deliberately conservative: when a transcript doesn't cleanly reduce, it returns `(False, None)` and the existing pipeline runs exactly as today. Precision over recall — a false positive here would (once live) silently mean a real event never gets a scene.
**SHIPPED IN SHADOW MODE ONLY.** `intelligence.extract_scenes` now computes the verdict alongside the existing `_is_garbage_transcript` / `transcript_too_short` gates and writes `chatter_classifier_verdict` (bool) and `chatter_classifier_reason` (str|None) onto the call doc — but does **not** skip scene extraction. Scene extraction runs exactly as it does today regardless of the verdict. The two fields are now surfaced in `/admin`'s correlation-debug `_call_summary`, same pattern as `corr_gate_veto` (#115/#126), so the next live measurement window can report the real-world false-positive rate before anything is wired to actually skip extraction.
## Backtest (offline, against the three existing dumps — see PR for script)
| dump | total calls | flagged chatter | false positives* |
|---|---|---|---|
| `corr_dump_9-7_0437am.json` (window #1) | 397 | 55 | **0** |
| `corr_dump_9-7_pm.json` (window #2) | 333 | 72 | **0** |
| `corr_dump_9-12.json` (window #3) | 272 | 27 | **0** |
| **total** | **1002** | **154** | **0** |
\* false positive = a flagged call carrying tags, coords, non-routine severity, or one of the specific transcripts the review docs named as dangerous to drop (the major "extinguishing fire" call, the geocoded 911-hangup call, both pursuit updates, the "6 Alpha... Pelham Station" subject check, the property-retrieval call, the Maronex subject check, the milepost-13.7 trespass call) — all individually re-checked and confirmed `(False, None)`.
Catch rate against a proxy chatter-universe (severity none/routine, no tags, no coords, no resolved incident_type — a superset of true housekeeping that also includes some mid-conversation follow-up fragments of real events, so this undercounts precision on the narrower true-chatter set): 55/343 (16%), 72/271 (27%), 27/201 (13%). Recall is intentionally low — the classifier only recognizes the two shapes above, not every housekeeping pattern (garbled spelled-out roll call like "Five units. Six hundred. Eighty-four. Ten-four.", ambiguous check-ins with a stray content word like "7-4, call Central. 6-9, location.", CID/plate/case-number readbacks that mix in other words) were deliberately left uncaught rather than risk a broader net.
## Tests
`drb-c2-core/tests/test_chatter_classifier.py` — real transcripts pulled from the dumps and review docs, both directions: chatter examples that must classify `True`, and the review docs' own named-dangerous real events that must classify `False`.
## Next step — NOT done here
**This is shadow mode only. Nothing skips scene extraction yet.** A `# TODO(server-26#<this-issue>)` at the call site marks exactly what has to be true before flipping it live: run at least one more live shadow-mode measurement window, confirm the false-positive rate against real production traffic is still zero (this backtest only proves it against three already-collected dumps), and only then wire the verdict to actually skip `extract_scenes`.
Links: #115, `CORRELATION_REVIEW_0907.md`, `CORRELATION_REVIEW_0907b.md`, `CORRELATION_REVIEW_0912.md`.
Review found and fixed two real gaps in the shadow-mode PR (#128, commit 3ae0bb2):
Placement bug (would have made the shadow data itself misleading): the classifier ran after the ≤5-word transcript_too_short skip, but 82% of its backtest flags were on ≤5-word transcripts — a live shadow window would have shown roughly a fifth of the real catch rate. Moved the classification ahead of that check, folded into the existing doc_set (no extra write).
No aggregate tally:admin.py's summary only had per-call fields, same gap corr_gate_veto (#126) fixed for the consensus gate. Added chatter_classifier_flagged/chatter_classifier_reason to the tally, spanning bothlinked calls and orphans — the target population (non-events) lands in both places, so linked alone undercounts it.
Confirmed still zero behavior change: nothing reads the verdict, extraction runs unconditionally either way. Sandboxed pytest 364 green (unchanged from the PR's own count — this was a placement/observability fix, not new logic).
Not fixed, flagged for the go-live decision instead of touched now: the review found the classifier's shapes are riskier than the offline backtest could show —
roll_call matches "roll call" anywhere in the transcript, not just whole-transcript ("Delayed for roll call, then respond to a structure fire at 12 Main Street" → flagged).
digit-hyphen street addresses match the bare-acknowledgement unit-ID pattern (the dumps already contain "72-Holland" — Whisper does emit this shape for real addresses).
some phonetic-alphabet callsign words are also real street names ("Central to 6 Henry, show me out at Lincoln and Union" → flagged).
All three are harmless in shadow mode (nothing gates on them) but mean a clean shadow-mode window is not sufficient evidence to flip to live — the TODO comment in intelligence.py now says so explicitly. Whoever does the live-flip work should tighten roll_call to whole-transcript and special-case digit-hyphen + phonetic-word-as-street-name before trusting a 0-false-positive window.
Review found and fixed two real gaps in the shadow-mode PR (#128, commit `3ae0bb2`):
1. **Placement bug (would have made the shadow data itself misleading):** the classifier ran *after* the ≤5-word `transcript_too_short` skip, but 82% of its backtest flags were on ≤5-word transcripts — a live shadow window would have shown roughly a fifth of the real catch rate. Moved the classification ahead of that check, folded into the existing `doc_set` (no extra write).
2. **No aggregate tally:** `admin.py`'s `summary` only had per-call fields, same gap `corr_gate_veto` (#126) fixed for the consensus gate. Added `chatter_classifier_flagged`/`chatter_classifier_reason` to the tally, spanning **both** `linked` calls and `orphans` — the target population (non-events) lands in both places, so `linked` alone undercounts it.
Confirmed still zero behavior change: nothing reads the verdict, extraction runs unconditionally either way. Sandboxed pytest 364 green (unchanged from the PR's own count — this was a placement/observability fix, not new logic).
**Not fixed, flagged for the go-live decision instead of touched now:** the review found the classifier's shapes are riskier than the offline backtest could show —
- `roll_call` matches "roll call" **anywhere** in the transcript, not just whole-transcript ("Delayed for roll call, then respond to a structure fire at 12 Main Street" → flagged).
- digit-hyphen street addresses match the bare-acknowledgement unit-ID pattern (the dumps already contain "72-Holland" — Whisper does emit this shape for real addresses).
- some phonetic-alphabet callsign words are also real street names ("Central to 6 Henry, show me out at Lincoln and Union" → flagged).
All three are harmless in shadow mode (nothing gates on them) but mean a clean shadow-mode window is not sufficient evidence to flip to live — the TODO comment in `intelligence.py` now says so explicitly. Whoever does the live-flip work should tighten `roll_call` to whole-transcript and special-case digit-hyphen + phonetic-word-as-street-name before trusting a 0-false-positive window.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Per #115's decision (comments 2026-09-08 and 2026-09-12) and both
CORRELATION_REVIEW_0907b.md/CORRELATION_REVIEW_0912.md: two consensus-layer fixes (#125, #126) have now both measured as no-ops on the top-line new-incident rate. The disagreement shape they target (corr_llm_action==orphan,corr_rules_action==new) is real but structurally small — backtesting a perfect version of that gate against window #3's own data caps out around 14/57 single-call "junk" incidents in a 4h window. 43/57 are outside the gate's scope entirely, and the largest single bucket (15/57) isagreed— both correlator tiers independently called it a new incident, with no disagreement for any consensus-layer fix to arbitrate.Decision (both review docs): stop iterating the consensus layer. Move the fix upstream — a dispatch-vs-chatter classifier in scene extraction (
intelligence.py) so radio housekeeping never produces a scene at all.What this issue ships
drb-c2-core/app/internal/chatter_classifier.py— a pure functionclassify_chatter(transcript) -> (bool, reason)that recognizes two shapes, drawn from hand-labeled examples in the review docs cross-referenced against the real dumps (not invented regexes):roll_call— transcript contains a roll-call announcement or line-up ("stand by for roll call", "Baker 1? Baker 1. Henry 1? ...")bare_acknowledgement— every token in the transcript reduces to a callsign/10-code/unit-ID (any digit-led token) or a small fixed set of procedural filler words ("central", "affirmative", "10-4", "stand by", "show me", "clear", ...). Any other word — a location, an action, a description — leaves the transcript unclassified.Deliberately conservative: when a transcript doesn't cleanly reduce, it returns
(False, None)and the existing pipeline runs exactly as today. Precision over recall — a false positive here would (once live) silently mean a real event never gets a scene.SHIPPED IN SHADOW MODE ONLY.
intelligence.extract_scenesnow computes the verdict alongside the existing_is_garbage_transcript/transcript_too_shortgates and writeschatter_classifier_verdict(bool) andchatter_classifier_reason(str|None) onto the call doc — but does not skip scene extraction. Scene extraction runs exactly as it does today regardless of the verdict. The two fields are now surfaced in/admin's correlation-debug_call_summary, same pattern ascorr_gate_veto(#115/#126), so the next live measurement window can report the real-world false-positive rate before anything is wired to actually skip extraction.Backtest (offline, against the three existing dumps — see PR for script)
corr_dump_9-7_0437am.json(window #1)corr_dump_9-7_pm.json(window #2)corr_dump_9-12.json(window #3)* false positive = a flagged call carrying tags, coords, non-routine severity, or one of the specific transcripts the review docs named as dangerous to drop (the major "extinguishing fire" call, the geocoded 911-hangup call, both pursuit updates, the "6 Alpha... Pelham Station" subject check, the property-retrieval call, the Maronex subject check, the milepost-13.7 trespass call) — all individually re-checked and confirmed
(False, None).Catch rate against a proxy chatter-universe (severity none/routine, no tags, no coords, no resolved incident_type — a superset of true housekeeping that also includes some mid-conversation follow-up fragments of real events, so this undercounts precision on the narrower true-chatter set): 55/343 (16%), 72/271 (27%), 27/201 (13%). Recall is intentionally low — the classifier only recognizes the two shapes above, not every housekeeping pattern (garbled spelled-out roll call like "Five units. Six hundred. Eighty-four. Ten-four.", ambiguous check-ins with a stray content word like "7-4, call Central. 6-9, location.", CID/plate/case-number readbacks that mix in other words) were deliberately left uncaught rather than risk a broader net.
Tests
drb-c2-core/tests/test_chatter_classifier.py— real transcripts pulled from the dumps and review docs, both directions: chatter examples that must classifyTrue, and the review docs' own named-dangerous real events that must classifyFalse.Next step — NOT done here
This is shadow mode only. Nothing skips scene extraction yet. A
# TODO(server-26#<this-issue>)at the call site marks exactly what has to be true before flipping it live: run at least one more live shadow-mode measurement window, confirm the false-positive rate against real production traffic is still zero (this backtest only proves it against three already-collected dumps), and only then wire the verdict to actually skipextract_scenes.Links: #115,
CORRELATION_REVIEW_0907.md,CORRELATION_REVIEW_0907b.md,CORRELATION_REVIEW_0912.md.Review found and fixed two real gaps in the shadow-mode PR (#128, commit
3ae0bb2):transcript_too_shortskip, but 82% of its backtest flags were on ≤5-word transcripts — a live shadow window would have shown roughly a fifth of the real catch rate. Moved the classification ahead of that check, folded into the existingdoc_set(no extra write).admin.py'ssummaryonly had per-call fields, same gapcorr_gate_veto(#126) fixed for the consensus gate. Addedchatter_classifier_flagged/chatter_classifier_reasonto the tally, spanning bothlinkedcalls andorphans— the target population (non-events) lands in both places, solinkedalone undercounts it.Confirmed still zero behavior change: nothing reads the verdict, extraction runs unconditionally either way. Sandboxed pytest 364 green (unchanged from the PR's own count — this was a placement/observability fix, not new logic).
Not fixed, flagged for the go-live decision instead of touched now: the review found the classifier's shapes are riskier than the offline backtest could show —
roll_callmatches "roll call" anywhere in the transcript, not just whole-transcript ("Delayed for roll call, then respond to a structure fire at 12 Main Street" → flagged).All three are harmless in shadow mode (nothing gates on them) but mean a clean shadow-mode window is not sufficient evidence to flip to live — the TODO comment in
intelligence.pynow says so explicitly. Whoever does the live-flip work should tightenroll_callto whole-transcript and special-case digit-hyphen + phonetic-word-as-street-name before trusting a 0-false-positive window.