correlator: fix consensus orphan-gate to test call substance, not empty corr_debug (#115)

The gate added in ca1d8fb checked rules_decision["corr_debug"] for a positive
signal, but that dict is empty at preview time for action=="new" (corr_path is
written at apply time). The check was always False, so the gate fired on real
events — replayed against corr_dump_9-7_pm.json it dropped ~36 linked calls
including a major "extinguishing fire", a moderate fire-alarm, geocoded calls
and pursuit updates.

Gate now runs against ctx (fully populated at preview time). It fires ONLY when
the call is substanceless: routine severity, no vehicle/geocode/tag, and no
incident already running on the same talkgroup. Any of those escalates to the
tiebreak instead. The substance predicate (has_event_substance) is factored out
of incident_correlator's creation gate and shared, so the two cannot diverge.

recorrelation_sweep: a call the gate parked gets a longer link-only retry budget
(10 vs 3) — the gate fires before any incident for the job exists, so the
substantive call that justifies linking can land after the standard ~6 min.
Still create_if_new=False.

incident_correlator location path: evaluate every in-radius candidate and link
the nearest that carries corroboration, instead of the first in an unsorted
`recent`. A unit-overlap location link is now tagged "location_unit_overlap" so
it stops merging into the fast path's bucket in the admin fit-signal histogram.

tests/test_consensus_gate.py: replaced the corr_debug-signal cases with ctx
substance cases (severity, coords, tags, vehicles, same-tg incident); added a
nearest-wins location test; the two location guard tests now assert they reach
the new guard. Full drb-c2-core suite 322 -> 325.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix

drb-c2-core/app/internal/incident_correlator.py

@@ -246,6 +246,22 @@ def _matching_units(call_units: Optional[list[str]], inc_units: Optional[list[st
     return [u for u in (call_units or []) if _normalize_unit(u) in inc_keys]
 
 
+def has_event_substance(ctx: dict) -> bool:
+    """
+    True when the call carries content beyond who-was-speaking-and-where:
+    a vehicle, a geocode, or a tag.
+
+    This is the substance half of the incident-creation gate (see
+    `_run_decision`, "Severity, not type, decides..."), factored out so the
+    consensus LLM-orphan gate in routers/upload.py mirrors it exactly and can
+    never drop a call the creation gate would have opened. `call_units` and
+    `location` are deliberately excluded — radio protocol puts a unit ID and a
+    place name in almost every transmission, so counting them as substance
+    makes the check trivially true.
+    """
+    return bool(ctx.get("call_vehicles") or ctx.get("coords") or ctx.get("tags"))
+
+
 def _infer_type_from_tags(tags: list[str]) -> Optional[str]:
     """Return an incident type inferred from tags, or None if ambiguous."""
     for tag in tags:
@@ -1180,6 +1196,10 @@ def _run_decision(ctx: dict) -> dict:
 
     # ── 2. Location path: proximity match (time-limited, cross-type) ─────────
     if not matched_incident and coords:
+        # server-26#115 — score every in-radius candidate and link the NEAREST
+        # that carries corroboration, rather than whichever incident happened to
+        # come first in an unsorted `recent`.
+        loc_candidates: list[tuple] = []
         for inc in recent:
             inc_coords = inc.get("location_coords")
             if not inc_coords:
@@ -1196,41 +1216,49 @@ def _run_decision(ctx: dict) -> dict:
                 elapsed_min = max(_incident_idle_minutes(inc, now), 0.1)
                 if (dist_km / elapsed_min) > _MAX_PURSUIT_SPEED_KM_PER_MIN:
                     continue  # implausible speed — skip this candidate
-            if dist_km <= radius:
-                # server-26#115 — a bare sub-radius distance is not enough on its
-                # own. Require corroboration: unit overlap with the candidate, OR
-                # a much tighter proximity. Pursuit incidents keep their
-                # movement-speed-validated wide radius (they passed the speed
-                # check above), so they are exempt.
-                unit_overlap = bool(
-                    _unit_keys(call_units) & _unit_keys(inc.get("units"))
-                )
-                tight_proximity = dist_km <= _LOCATION_TIGHT_PROXIMITY_KM
-                if not (is_pursuit_inc or unit_overlap or tight_proximity):
-                    logger.info(
-                        f"Correlator location-path skipped: call {call_id} vs "
-                        f"{inc['incident_id']} — dist={dist_km:.2f}km within radius "
-                        f"but no unit overlap and not tight-proximity "
-                        f"(<= {_LOCATION_TIGHT_PROXIMITY_KM}km)"
-                    )
-                    continue
-                matched_incident = inc
-                fit_signal = "unit_overlap" if unit_overlap else "location_proximity"
-                corr_debug = {
-                    "corr_path": "location",
-                    "corr_distance_km": round(dist_km, 3),
-                    "corr_pursuit_mode": is_pursuit_inc,
-                    "corr_fit_signal": fit_signal,
-                }
-                if unit_overlap and call_units:
-                    corr_debug["corr_matched_units"] = _matching_units(
-                        call_units, inc.get("units")
-                    )
+            if dist_km > radius:
+                continue
+            # server-26#115 — a bare sub-radius distance is not enough on its
+            # own. Require corroboration: unit overlap with the candidate, OR
+            # a much tighter proximity. Pursuit incidents keep their
+            # movement-speed-validated wide radius (they passed the speed
+            # check above), so they are exempt.
+            unit_overlap = bool(
+                _unit_keys(call_units) & _unit_keys(inc.get("units"))
+            )
+            tight_proximity = dist_km <= _LOCATION_TIGHT_PROXIMITY_KM
+            if not (is_pursuit_inc or unit_overlap or tight_proximity):
                 logger.info(
-                    f"Correlator location-path: call {call_id} → {inc['incident_id']} "
-                    f"(dist={dist_km:.2f}km, pursuit={is_pursuit_inc}, signal={fit_signal})"
+                    f"Correlator location-path skipped: call {call_id} vs "
+                    f"{inc['incident_id']} — dist={dist_km:.2f}km within radius "
+                    f"but no unit overlap and not tight-proximity "
+                    f"(<= {_LOCATION_TIGHT_PROXIMITY_KM}km)"
                 )
-                break
+                continue
+            loc_candidates.append((dist_km, unit_overlap, is_pursuit_inc, inc))
+
+        if loc_candidates:
+            loc_candidates.sort(key=lambda c: c[0])
+            dist_km, unit_overlap, is_pursuit_inc, inc = loc_candidates[0]
+            matched_incident = inc
+            # Distinct from the fast path's "unit_overlap" so the admin
+            # corr_fit_signal histogram (routers/admin.py) does not merge a
+            # location-path link into the fast-path bucket (#35).
+            fit_signal = "location_unit_overlap" if unit_overlap else "location_proximity"
+            corr_debug = {
+                "corr_path": "location",
+                "corr_distance_km": round(dist_km, 3),
+                "corr_pursuit_mode": is_pursuit_inc,
+                "corr_fit_signal": fit_signal,
+            }
+            if unit_overlap and call_units:
+                corr_debug["corr_matched_units"] = _matching_units(
+                    call_units, inc.get("units")
+                )
+            logger.info(
+                f"Correlator location-path: call {call_id} → {inc['incident_id']} "
+                f"(dist={dist_km:.2f}km, pursuit={is_pursuit_inc}, signal={fit_signal})"
+            )
 
     # ── 2.5. Cross-TG path: same department, overlapping units, moderate similarity ──
     #
@@ -1367,7 +1395,7 @@ def _run_decision(ctx: dict) -> dict:
     # each.  A vehicle, a geocode, or a tag means the extractor found something
     # beyond who was speaking and where they stood.
     if not resolved_type:
-        has_substance = bool(call_vehicles or coords or tags)
+        has_substance = has_event_substance(ctx)
         if call_severity in ("minor", "moderate", "major") or has_substance:
             resolved_type = "other"
             logger.info(

drb-c2-core/app/internal/recorrelation_sweep.py

@@ -20,6 +20,22 @@ from app.internal.logger import logger
 from app.internal import firestore as fstore
 from app.config import settings
 
+# Standard link-only retry budget before a call is tombstoned corr_path="unlinked".
+MAX_SWEEP_ATTEMPTS = 3
+# server-26#115 — a call the consensus LLM-orphan gate parked (llm=orphan vs
+# rules=new, no substance) gets a longer budget. The gate fires before any
+# incident for the job may exist, so the substantive call that would justify
+# linking can land well after the standard ~6 min. Still link-only: a genuinely
+# thin call must not mint an incident, and the rules creation gate would re-orphan
+# it anyway.
+GATED_ORPHAN_SWEEP_ATTEMPTS = 10
+
+
+def _max_sweep_attempts(call: dict) -> int:
+    if call.get("corr_consensus") == "llm_orphan_gate":
+        return GATED_ORPHAN_SWEEP_ATTEMPTS
+    return MAX_SWEEP_ATTEMPTS
+
 
 async def recorrelation_loop() -> None:
     interval = settings.summary_interval_minutes * 60
@@ -46,10 +62,9 @@ async def _run_sweep_pass() -> None:
         ("status",   "==",  "ended"),
         ("ended_at", ">=",  cutoff),
     ])
-    # corr_path="unlinked" is written after MAX_SWEEP_ATTEMPTS failures.
+    # corr_path="unlinked" is written after the attempt budget is exhausted.
     # Allows a few retries so a welfare-check call can link to an escalation
     # incident that is created a few minutes later, without sweeping 30× forever.
-    MAX_SWEEP_ATTEMPTS = 3
     orphans = [
         c for c in recent_ended
         if not c.get("incident_ids") and not c.get("incident_id")
@@ -61,7 +76,7 @@ async def _run_sweep_pass() -> None:
         # the thin path minutes later and attached to whatever was most recent —
         # a second route into the over-merge the thin fix above addresses.
         and not c.get("skip_reason")
-        and c.get("corr_sweep_count", 0) < MAX_SWEEP_ATTEMPTS
+        and c.get("corr_sweep_count", 0) < _max_sweep_attempts(c)
     ]
 
     if not orphans:
@@ -120,12 +135,12 @@ async def _recorrelate_orphan(call: dict) -> bool:
         )
         return True
 
-    # Increment the attempt counter. Once MAX_SWEEP_ATTEMPTS is reached the
-    # orphan filter above will stop picking this call up, and we write
-    # corr_path="unlinked" as a permanent tombstone.
+    # Increment the attempt counter. Once the budget is reached the orphan filter
+    # above will stop picking this call up, and we write corr_path="unlinked" as
+    # a permanent tombstone.
     attempts = call.get("corr_sweep_count", 0) + 1
     update: dict = {"corr_sweep_count": attempts}
-    if attempts >= 3:
+    if attempts >= _max_sweep_attempts(call):
         update["corr_path"] = "unlinked"
     await fstore.doc_set("calls", call_id, update)
     return False