correlation: rules decide, one cheap LLM call only when ambiguous; drop the consensus tiebreak (server-26#299)

The rules engine now commits every confident decision alone. _run_decision
marks a decision ambiguous (new_with_candidates, unfit_tg_candidate,
unverified_unit_link) and only then does upload._correlate make ONE
llm_correlator.decide call; a usable answer is committed, otherwise the rules
decision stands. Removed: _correlate_with_consensus, llm_correlator.tiebreak /
decisions_agree / tiebreak prompt, corr_smart_model, the correlation_smart
health tier, the LLM-orphan gate (_call_is_substanceless, effectively dead:
it could only fire on minor-severity untyped calls) and its sweep budget.
corr_consensus is now rules_only | llm | rules_fallback | rules_guard, with
corr_band naming why; admin debug and replay metrics read the new values.

Section fixes in the same pass:
- #115: no tiebreaker left to manufacture incidents; an LLM orphan of a
  rules "new" is honoured only for untyped, substanceless, routine/minor calls.
- #86: divergence veto bar moved to config (embedding_divergence_threshold);
  the band between it and the match bar is ambiguous, not a silent merge.
- #103: a unit link with no embedding is "unit_overlap_unverified" ->
  ambiguous, instead of skipping the veto and linking as verified.
- #189: unit-only and LLM links refused once an incident is open 90 min;
  unit links on incidents open >30 min are ambiguous; LLM links distance-
  guarded (incident_max_spread_km) and only to shown candidates.
- #190: severity steps down on an explicit stand-down phrase; new
  severity_peak keeps the monotonic max.
- #88: parse_llm_bool fails closed on "false"; scene closures honoured only
  on confident links or the incident the scene opened.
- #27: sweep budget is the scan window (wall clock), not 3 attempts.
- LLM prompt: "Reserve new" bullet no longer contradicts the #189 EXCEPT.
- updated_at never moves backward on a back-dated link.

Unmeasured: no live AI or replay (credits unfunded); band frequency, F1 and
thresholds are unverified. Must be re-scored on both answer keys in Replay
before merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Logan Cusano
2026-10-27 20:26:38 -04:00
co-authored by Claude Opus 5.5
parent f4d78aba77
commit 3246fc97b0
18 changed files with 1207 additions and 1062 deletions
+39 -16
View File
@@ -70,20 +70,16 @@ class Settings(BaseSettings):
# Gemini (intelligence extraction, embeddings, incident summaries)
gemini_api_key: Optional[str] = None
# Correlation consensus models
# corr_cheap_model — first-pass LLM correlator (runs on every call)
# corr_smart_model — tiebreaker (only fires when rules and cheap LLM disagree)
# Both IDs below were retired by Google and returned 404 on every call from
# some point before 2026-08-18 until they were corrected. Because a failed
# LLM call falls back to the rules decision, nothing broke loudly -- the
# entire LLM tier and the consensus tiebreak were simply dead in production
# while correlation behaviour was being tuned against rules-only output.
# Verify against https://ai.google.dev/gemini-api/docs/models before changing.
# Correlation LLM (server-26#299). The rules engine decides every confident
# case; this model is called at most ONCE per scene, and only when the rules
# flag the decision as ambiguous (incident_correlator._ambiguity). There is
# no second "smart" tier any more: the cheap/smart consensus tiebreak ran on
# ~15% of scenes and resolved nearly every disagreement as "new" (#115).
# The model ID was once retired by Google and 404'd on every call for days;
# a failed LLM call falls back to the rules decision, so nothing broke
# loudly. Verify against https://ai.google.dev/gemini-api/docs/models
# before changing.
corr_cheap_model: str = "gemini-3.6-flash" # was gemini-2.0-flash (shut down)
# gemini-2.5-pro was closed to new projects by 2026-09 (every tiebreak 404'd
# in the first replay run, server-26#170); Google lists no stable Pro model,
# so the smart tier is the newest stable Flash instead.
corr_smart_model: str = "gemini-3.8-flash" # was gemini-2.5-pro, gemini-1.5-pro
# Transcript correction (server-26#36). Runs inside transcription, once per
# transcribed call above MIN_WORDS_FOR_CORRECTION, so it is priced like STT
# rather than like the correlation tier — cheap model on purpose.
@@ -122,6 +118,35 @@ class Settings(BaseSettings):
embedding_similarity_threshold: float = 0.93 # slow-path: requires location corroboration
embedding_no_location_threshold: float = 0.97 # slow-path: match without location (very high bar)
embedding_cross_tg_threshold: float = 0.85 # cross-TG path: same dept + 2+ shared units
# server-26#86: the content-divergence veto on a unit-overlap link. This is
# a DIVERGENCE bar, not a match bar, so it sits deliberately below
# embedding_similarity_threshold: two calls about different jobs on one
# dispatch channel still share ten-codes, unit IDs and address phrasing, so
# "not a match" (< 0.93) is not evidence of "a different job". Below this
# bar the link is vetoed; between the two bars the link is neither
# confirmed nor refused, so it is AMBIGUOUS and goes to the LLM (#299)
# instead of silently merging. 0.82 is the historical literal, unmeasured
# against real score distributions.
embedding_divergence_threshold: float = 0.82
# A unit-overlap link is only a confident one while the incident is fresh:
# idle under this many minutes (the old hard-coded 15 of the divergence
# veto) AND open under unit_link_confident_max_span_minutes. Past either,
# shared units alone are mechanism B (CORRELATION_REVIEW_0820.md) — patrol
# units work job after job on one channel — so the link is ambiguous.
unit_link_fresh_idle_minutes: int = 15
unit_link_confident_max_span_minutes: int = 30
# server-26#189: an incident open this long accepts no more WEAK links
# (unit overlap without location/content agreement, or an LLM link) — only
# vehicle, location or embedding evidence, until the hard
# incident_max_duration_minutes cap. The 09-27 "Rollover Entrapment"
# sink absorbed unrelated Ch 1 jobs for 2h34m this way. Thin replies are
# exempt: they never refresh updated_at, so they cannot keep a sink alive.
incident_weak_link_max_span_minutes: int = 90
# server-26#189: an LLM link may not pull a call geocoded farther than
# this from a non-pursuit incident's verified pin (the same 4x
# location_proximity_km the slow path accepts). The LLM tier had no
# distance check at all, which is how one incident drew a 37 km path.
incident_max_spread_km: float = 2.0
location_proximity_km: float = 0.5 # radius for location-proximity matching
geocode_max_km: float = 40.0 # reject geocode results farther than this from the node
incident_auto_resolve_minutes: int = 90 # auto-resolve after N minutes with no new calls (major / unknown severity)
@@ -147,9 +172,7 @@ class Settings(BaseSettings):
# at 9.7 min. Every correct thin attach in that dump was <= 3.4 min idle
# and every wrong one was >= 8.2, so 5 separates them with room on both
# sides. Genuine back-and-forth is handled by the 30-second tier-1 path
# above this. Also the escape hatch in routers/upload.py's LLM-orphan gate
# (_recent_incident_on_same_talkgroup, server-26#115) — check both call
# sites before retuning this.
# above this.
tg_dispatch_thin_idle_minutes: int = 5
# ── Hard caps: an incident past either of these stops accepting calls ──────