Merge feat/297-cheaper-ai into feat/299-correlation-simplify

Resolve upload.py (no-scene fallback uses the new _correlate with #298's
non_event_chatter reason) and chatter_classifier docstring (gate built, off by
default). Fold intelligence._llm_bool into incident_correlator.parse_llm_bool (#88).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Logan Cusano
2026-10-27 20:31:54 -04:00
co-authored by Claude Opus 5.5
22 changed files with 2046 additions and 231 deletions
+45 -7
View File
@@ -14,13 +14,18 @@ This module is that classifier. It is a PURE function of the transcript text —
no Firestore, no LLM call, no side effects — so it is cheap to run on every
transcript and cheap to test against real dumps offline.
SHADOW MODE ONLY. As of this module's introduction, nothing skips scene
extraction based on this verdict. `intelligence.extract_scenes` calls
`classify_chatter` purely to record the verdict on the call doc
(`chatter_classifier_verdict` / `chatter_classifier_reason`) so it becomes
observable in the next `/admin` correlation-debug dump (as `corr_gate_veto`
was, before server-26#299 removed that gate). See the TODO at that call site for
what has to be true before this flips live.
A COST GATE SINCE server-26#298, through `gate_chatter`, not
`classify_chatter` — built but OFF by default (`chatter_gate_enabled`) until a
shadow window shows gated calls don't over-merge. `classify_chatter` is still
the shadow verdict recorded on every call doc (`chatter_classifier_verdict` /
`chatter_classifier_reason`), so the `/admin` tallies keep measuring the same
thing. The gate is stricter on the one shape the #127 review flagged as a live
false-positive risk: `roll_call` matches "roll call" anywhere, so "Delayed for
roll call, then respond to a structure fire at 12 Main Street" is a roll_call
verdict. The gate only skips a roll call whose remaining words are callsigns
and roll-call phrasing. When enabled, gated transcripts skip transcript
correction and scene extraction (and with it embedding and geocoding); the free
rule paths (clearance parse, self-initiated/plate backstop) still run.
Precision over recall, deliberately. A false positive here — flagging a REAL
event as chatter — would, once live, silently mean that event never gets a
@@ -133,3 +138,36 @@ def classify_chatter(transcript: Optional[str]) -> tuple[bool, Optional[str]]:
return False, None
return True, "bare_acknowledgement"
# Words a roll call is made of, beyond the filler/designator/phonetic sets.
_ROLL_CALL_WORDS = frozenset({
"roll", "call", "all", "cars", "car", "patrol", "patrols", "troop", "staff",
"concludes", "time", "came", "because", "volunteer", "nyc", "kl", "henry",
"here", "present", "s", "m", "it",
})
# A roll call may name a place or two ("Post 4, Ossining"); more than this many
# leftover words means something else is being said in the same transmission.
_ROLL_CALL_MAX_LEFTOVER = 2
def gate_chatter(transcript: Optional[str]) -> tuple[bool, Optional[str]]:
"""
Whether to skip paid AI for this transcript (server-26#298).
`classify_chatter`'s verdict, with roll_call narrowed: the transcript must
be roll-call content end to end, not a roll call mentioned on the way to
something real. Precision over recall still — a false positive here means
a real event never gets a scene.
"""
is_chatter, reason = classify_chatter(transcript)
if not is_chatter:
return False, None
if reason == "roll_call":
leftover = [
t for t in _tokenize(transcript or "")
if not _is_filler_token(t) and t not in _ROLL_CALL_WORDS
]
if len(leftover) > _ROLL_CALL_MAX_LEFTOVER:
return False, None
return True, reason