correlator: gate LLM-orphan against rules-new instead of escalating to tiebreak (#115)
CORRELATION_REVIEW_0907b.md measured that radio housekeeping (unit check-ins, roll call, 10-8/10-98 clearings) is being promoted to incidents. Every case reads corr_llm_action=orphan, corr_rules_action=new, corr_consensus=tiebreak -> new: the cheap LLM correctly reads "not an incident", the rules engine says `new` only because there is no incident to link to, and the smart tiebreaker then sides with rules ~21/21. Reframing the tiebreaker prompt (#116) did nothing. The fix is a consensus-logic gate, not another prompt. Fix 1 (routers/upload.py) - LLM-orphan gate in _correlate_with_consensus: when the cheap LLM says `orphan` and the rules engine says `new` with NO positive event signal, resolve to `orphan` and skip the tiebreak call entirely. "No positive signal" = the rules corr_debug carries neither a positive corr_path (unit-continuity / location / fast/disambig / fast/single) nor a positive corr_fit_signal (unit_overlap / location_proximity). When it does carry one, the existing escalation-to-tiebreak is kept so a genuine event the LLM misreads as orphan still gets the second look. The resolved outcome records corr_consensus="llm_orphan_gate" (greppable, distinct from "tiebreak") and keeps corr_llm_reasoning / corr_rules_action / corr_llm_action populated. Fix 2 (incident_correlator.py) - tighten corr_path=location: the location path linked on a bare sub-location_proximity_km (0.5 km) distance with no unit or content check, which stitched a vehicle lockout to a station-restroom slip and merged two different churches an hour apart. A location link now requires unit overlap with the candidate OR a distance under a tighter bar (_LOCATION_TIGHT_PROXIMITY_KM = 0.2 km). Pursuit incidents keep their movement-speed-validated wide radius. A surviving location link now also writes corr_fit_signal (unit_overlap | location_proximity), consistent with Fix 1's positive-signal set. Tests: new tests/test_consensus_gate.py (13 cases) - the gate resolves to orphan without calling tiebreak on a no-signal disagreement; a unit_overlap / location_proximity / unit-continuity / fast-disambig rules signal still escalates; llm=link vs rules=new still escalates; the location path drops a shared-area candidate with neither unit overlap nor tight proximity, links on unit overlap, and links on tight proximity alone. Full c2-core suite 309 -> 322 passing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
7f4d684966
commit
ca1d8fbdae
@@ -100,6 +100,30 @@ async def upload_call_audio(
|
||||
return {"url": gcs_uri}
|
||||
|
||||
|
||||
# server-26#115 — a rules "new" only counts as a real "this is an event" verdict
|
||||
# when it carries one of these signals. A bare "new" (no link candidate found)
|
||||
# is trivially true for radio housekeeping (check-ins, roll call, 10-8/10-98) and
|
||||
# must not out-vote a cheap-LLM "orphan" that has actually read the transcript.
|
||||
_POSITIVE_CORR_PATHS = frozenset({
|
||||
"unit-continuity", "location", "fast/disambig", "fast/single",
|
||||
})
|
||||
_POSITIVE_FIT_SIGNALS = frozenset({"unit_overlap", "location_proximity"})
|
||||
|
||||
|
||||
def _rules_has_positive_event_signal(rules_decision: dict) -> bool:
|
||||
"""
|
||||
True when the rules engine's decision carries a positive "this is an event"
|
||||
signal (unit overlap, location proximity, or a continuity/disambiguation
|
||||
path) rather than merely "no incident to link to".
|
||||
"""
|
||||
dbg = rules_decision.get("corr_debug") or {}
|
||||
if dbg.get("corr_fit_signal") in _POSITIVE_FIT_SIGNALS:
|
||||
return True
|
||||
if dbg.get("corr_path") in _POSITIVE_CORR_PATHS:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
async def _correlate_with_consensus(
|
||||
call_id: str,
|
||||
node_id: str,
|
||||
@@ -151,6 +175,38 @@ async def _correlate_with_consensus(
|
||||
rules_decision["corr_debug"]["corr_llm_reasoning"] = llm_decision.get("reasoning", "")
|
||||
return await incident_correlator.apply_correlation(preview)
|
||||
|
||||
# server-26#115 — LLM-orphan gate.
|
||||
# When the cheap LLM says `orphan` and the rules engine says `new` with NO
|
||||
# positive event signal (i.e. rules only found "nothing to link to" — trivially
|
||||
# true for radio housekeeping), resolve to `orphan` and DO NOT pay for the
|
||||
# smart tiebreaker. The LLM has read the transcript; a bare rules `new` has
|
||||
# not, and the tiebreaker sided with rules ~21/21 of the time on exactly this
|
||||
# disagreement (CORRELATION_REVIEW_0907b.md). A genuine event the LLM misreads
|
||||
# as orphan still escalates, because the rules result then carries a real
|
||||
# signal (unit overlap, location proximity, unit-continuity / disambig).
|
||||
if (
|
||||
llm_decision["action"] == "orphan"
|
||||
and rules_decision["action"] == "new"
|
||||
and not _rules_has_positive_event_signal(rules_decision)
|
||||
):
|
||||
logger.info(
|
||||
f"Consensus gate for call {call_id}: llm=orphan vs rules=new with no "
|
||||
f"positive rules signal — resolving orphan, skipping tiebreak"
|
||||
)
|
||||
gated = {
|
||||
"action": "orphan",
|
||||
"matched_incident": None,
|
||||
"incident_type": None,
|
||||
"corr_debug": dict(rules_decision.get("corr_debug") or {}),
|
||||
}
|
||||
gated["corr_debug"].update({
|
||||
"corr_consensus": "llm_orphan_gate",
|
||||
"corr_rules_action": rules_decision["action"],
|
||||
"corr_llm_action": llm_decision["action"],
|
||||
"corr_llm_reasoning": llm_decision.get("reasoning", ""),
|
||||
})
|
||||
return await incident_correlator.apply_correlation({"decision": gated, "ctx": ctx})
|
||||
|
||||
# Disagree — escalate to the smarter tiebreaker.
|
||||
logger.info(
|
||||
f"Consensus disagreement for call {call_id}: "
|
||||
|
||||
Reference in New Issue
Block a user