correlation: rules decide, one cheap LLM call only when ambiguous; drop the consensus tiebreak (server-26#299)
The rules engine now commits every confident decision alone. _run_decision marks a decision ambiguous (new_with_candidates, unfit_tg_candidate, unverified_unit_link) and only then does upload._correlate make ONE llm_correlator.decide call; a usable answer is committed, otherwise the rules decision stands. Removed: _correlate_with_consensus, llm_correlator.tiebreak / decisions_agree / tiebreak prompt, corr_smart_model, the correlation_smart health tier, the LLM-orphan gate (_call_is_substanceless, effectively dead: it could only fire on minor-severity untyped calls) and its sweep budget. corr_consensus is now rules_only | llm | rules_fallback | rules_guard, with corr_band naming why; admin debug and replay metrics read the new values. Section fixes in the same pass: - #115: no tiebreaker left to manufacture incidents; an LLM orphan of a rules "new" is honoured only for untyped, substanceless, routine/minor calls. - #86: divergence veto bar moved to config (embedding_divergence_threshold); the band between it and the match bar is ambiguous, not a silent merge. - #103: a unit link with no embedding is "unit_overlap_unverified" -> ambiguous, instead of skipping the veto and linking as verified. - #189: unit-only and LLM links refused once an incident is open 90 min; unit links on incidents open >30 min are ambiguous; LLM links distance- guarded (incident_max_spread_km) and only to shown candidates. - #190: severity steps down on an explicit stand-down phrase; new severity_peak keeps the monotonic max. - #88: parse_llm_bool fails closed on "false"; scene closures honoured only on confident links or the incident the scene opened. - #27: sweep budget is the scan window (wall clock), not 3 attempts. - LLM prompt: "Reserve new" bullet no longer contradicts the #189 EXCEPT. - updated_at never moves backward on a back-dated link. Unmeasured: no live AI or replay (credits unfunded); band frequency, F1 and thresholds are unverified. Must be re-scored on both answer keys in Replay before merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
f4d78aba77
commit
3246fc97b0
+39
-16
@@ -70,20 +70,16 @@ class Settings(BaseSettings):
|
||||
|
||||
# Gemini (intelligence extraction, embeddings, incident summaries)
|
||||
gemini_api_key: Optional[str] = None
|
||||
# Correlation consensus models
|
||||
# corr_cheap_model — first-pass LLM correlator (runs on every call)
|
||||
# corr_smart_model — tiebreaker (only fires when rules and cheap LLM disagree)
|
||||
# Both IDs below were retired by Google and returned 404 on every call from
|
||||
# some point before 2026-08-18 until they were corrected. Because a failed
|
||||
# LLM call falls back to the rules decision, nothing broke loudly -- the
|
||||
# entire LLM tier and the consensus tiebreak were simply dead in production
|
||||
# while correlation behaviour was being tuned against rules-only output.
|
||||
# Verify against https://ai.google.dev/gemini-api/docs/models before changing.
|
||||
# Correlation LLM (server-26#299). The rules engine decides every confident
|
||||
# case; this model is called at most ONCE per scene, and only when the rules
|
||||
# flag the decision as ambiguous (incident_correlator._ambiguity). There is
|
||||
# no second "smart" tier any more: the cheap/smart consensus tiebreak ran on
|
||||
# ~15% of scenes and resolved nearly every disagreement as "new" (#115).
|
||||
# The model ID was once retired by Google and 404'd on every call for days;
|
||||
# a failed LLM call falls back to the rules decision, so nothing broke
|
||||
# loudly. Verify against https://ai.google.dev/gemini-api/docs/models
|
||||
# before changing.
|
||||
corr_cheap_model: str = "gemini-3.6-flash" # was gemini-2.0-flash (shut down)
|
||||
# gemini-2.5-pro was closed to new projects by 2026-09 (every tiebreak 404'd
|
||||
# in the first replay run, server-26#170); Google lists no stable Pro model,
|
||||
# so the smart tier is the newest stable Flash instead.
|
||||
corr_smart_model: str = "gemini-3.8-flash" # was gemini-2.5-pro, gemini-1.5-pro
|
||||
# Transcript correction (server-26#36). Runs inside transcription, once per
|
||||
# transcribed call above MIN_WORDS_FOR_CORRECTION, so it is priced like STT
|
||||
# rather than like the correlation tier — cheap model on purpose.
|
||||
@@ -122,6 +118,35 @@ class Settings(BaseSettings):
|
||||
embedding_similarity_threshold: float = 0.93 # slow-path: requires location corroboration
|
||||
embedding_no_location_threshold: float = 0.97 # slow-path: match without location (very high bar)
|
||||
embedding_cross_tg_threshold: float = 0.85 # cross-TG path: same dept + 2+ shared units
|
||||
# server-26#86: the content-divergence veto on a unit-overlap link. This is
|
||||
# a DIVERGENCE bar, not a match bar, so it sits deliberately below
|
||||
# embedding_similarity_threshold: two calls about different jobs on one
|
||||
# dispatch channel still share ten-codes, unit IDs and address phrasing, so
|
||||
# "not a match" (< 0.93) is not evidence of "a different job". Below this
|
||||
# bar the link is vetoed; between the two bars the link is neither
|
||||
# confirmed nor refused, so it is AMBIGUOUS and goes to the LLM (#299)
|
||||
# instead of silently merging. 0.82 is the historical literal, unmeasured
|
||||
# against real score distributions.
|
||||
embedding_divergence_threshold: float = 0.82
|
||||
# A unit-overlap link is only a confident one while the incident is fresh:
|
||||
# idle under this many minutes (the old hard-coded 15 of the divergence
|
||||
# veto) AND open under unit_link_confident_max_span_minutes. Past either,
|
||||
# shared units alone are mechanism B (CORRELATION_REVIEW_0820.md) — patrol
|
||||
# units work job after job on one channel — so the link is ambiguous.
|
||||
unit_link_fresh_idle_minutes: int = 15
|
||||
unit_link_confident_max_span_minutes: int = 30
|
||||
# server-26#189: an incident open this long accepts no more WEAK links
|
||||
# (unit overlap without location/content agreement, or an LLM link) — only
|
||||
# vehicle, location or embedding evidence, until the hard
|
||||
# incident_max_duration_minutes cap. The 09-27 "Rollover Entrapment"
|
||||
# sink absorbed unrelated Ch 1 jobs for 2h34m this way. Thin replies are
|
||||
# exempt: they never refresh updated_at, so they cannot keep a sink alive.
|
||||
incident_weak_link_max_span_minutes: int = 90
|
||||
# server-26#189: an LLM link may not pull a call geocoded farther than
|
||||
# this from a non-pursuit incident's verified pin (the same 4x
|
||||
# location_proximity_km the slow path accepts). The LLM tier had no
|
||||
# distance check at all, which is how one incident drew a 37 km path.
|
||||
incident_max_spread_km: float = 2.0
|
||||
location_proximity_km: float = 0.5 # radius for location-proximity matching
|
||||
geocode_max_km: float = 40.0 # reject geocode results farther than this from the node
|
||||
incident_auto_resolve_minutes: int = 90 # auto-resolve after N minutes with no new calls (major / unknown severity)
|
||||
@@ -147,9 +172,7 @@ class Settings(BaseSettings):
|
||||
# at 9.7 min. Every correct thin attach in that dump was <= 3.4 min idle
|
||||
# and every wrong one was >= 8.2, so 5 separates them with room on both
|
||||
# sides. Genuine back-and-forth is handled by the 30-second tier-1 path
|
||||
# above this. Also the escape hatch in routers/upload.py's LLM-orphan gate
|
||||
# (_recent_incident_on_same_talkgroup, server-26#115) — check both call
|
||||
# sites before retuning this.
|
||||
# above this.
|
||||
tg_dispatch_thin_idle_minutes: int = 5
|
||||
|
||||
# ── Hard caps: an incident past either of these stops accepting calls ──────
|
||||
|
||||
Reference in New Issue
Block a user