Compare unit IDs by normalised key, not exact string
Build & Deploy / Build & push images (push) Successful in 4m6s
Build & Deploy / Deploy to VM (push) Successful in 2m32s

Dispatch audio names the same unit several ways within one conversation, and
every comparison in the correlator used exact string equality, so a follow-up
transmission from a unit already on an incident simply failed to find it. With
the creation gate no longer letting routine traffic open its own incident,
these stopped becoming junk incidents and started becoming orphans instead --
which is how they became visible. In the 01:05Z dump, five of eighteen orphans
were calls belonging to an incident that was open at that moment:

    "K-9A2"     vs "K-9-A-2"     punctuation
    "5-1-6"     vs "516"         digits read out individually
    "37"        vs "37th Post"   ordinal plus role word
    "11-Victor" vs "11 Victor"   hyphen vs space

_normalize_unit lowercases, drops punctuation and role words (post/unit/car),
strips ordinal suffixes, and joins the remaining tokens, so each pair above
collapses to one key. All six comparison sites now go through it: the two
fast-path debug reporters, unit-continuity candidate selection and its
reassignment check, the cross-talkgroup 2+ shared-unit test, and the
disambiguation scorer.

What it deliberately does NOT do is match a bare district letter -- "Adam" is
not treated as "6-Adam". Every district has an Adam, and collapsing them would
merge unrelated incidents across districts. That leaves a couple of the
observed orphans unlinked, which is the right trade: a missed link leaves an
orphan the re-correlation sweep retries three times, while a false link
corrupts an incident permanently and nothing walks it back.

Two smaller things fall out of the shared helper. Matches are reported as the
original spoken strings rather than the normalised keys, so corr_matched_units
stays readable in the debug view. And a unit made only of role words ("Post")
would normalise to the empty string and then compare equal to every other such
unit, so it falls back to the raw text -- tested, because that failure would be
silent and would merge aggressively.

Adds 13 cases: each observed pair, five pairs that must stay distinct, the
empty-key guard, match reporting, and an end-to-end check that the K-9A2 call
now links where it previously orphaned. 38 pass.

No new environment variables, so CI deploys this without an ansible run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Logan Cusano
2026-08-16 21:35:06 -04:00
co-authored by Claude Opus 5
parent 94ce9d48e2
commit c09cb72f66
2 changed files with 128 additions and 15 deletions
+60 -1
View File
@@ -16,7 +16,9 @@ Firestore. _update_incident writes, so its test patches fstore.
import pytest
from datetime import datetime, timedelta, timezone
from unittest.mock import AsyncMock, patch
from app.internal.incident_correlator import _run_decision, _update_incident
from app.internal.incident_correlator import (
_run_decision, _update_incident, _normalize_unit, _matching_units,
)
NOW = datetime(2026, 8, 16, 21, 0, 0, tzinfo=timezone.utc)
@@ -188,3 +190,60 @@ async def test_substantive_link_does_refresh_updated_at():
updates = mock_fstore.doc_set.await_args.args[2]
assert updates["updated_at"] == NOW.isoformat()
assert "last_thin_at" not in updates
# ---------------------------------------------------------------------------
# Unit-ID normalisation — dispatch names the same unit several ways
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("spoken,other", [
("K-9A2", "K-9-A-2"), # punctuation only
("5-1-6", "516"), # digits read out individually
("37", "37th Post"), # ordinal + role word
("11-Victor", "11 Victor"), # hyphen vs space
("Post 5", "5"), # bare role word
("post 1-2", "Post 1-2"), # case
])
def test_same_unit_spoken_differently_normalises_alike(spoken, other):
"""Every pair here was observed as one real unit failing to match itself."""
assert _normalize_unit(spoken) == _normalize_unit(other)
@pytest.mark.parametrize("a,b", [
("6-Adam", "Adam"), # every district has an Adam — must stay distinct
("6-Adam", "7-Adam"),
("11-Victor", "11-Xray"),
("516", "517"),
("3", "39"),
])
def test_genuinely_different_units_stay_distinct(a, b):
assert _normalize_unit(a) != _normalize_unit(b)
def test_role_only_unit_does_not_collapse_to_empty():
"""
"Post" is all noise words. Normalising it to "" would make every such unit
equal to every other, so it falls back to the raw text instead.
"""
assert _normalize_unit("Post") != ""
assert _normalize_unit("Post") != _normalize_unit("Unit")
def test_matching_units_reports_the_original_spoken_strings():
"""Debug output has to stay readable, so matches come back un-normalised."""
assert _matching_units(["K-9A2", "6-Adam"], ["K-9-A-2"]) == ["K-9A2"]
def test_matching_units_empty_when_nothing_overlaps():
assert _matching_units(["6-Adam"], ["7-Adam", "516"]) == []
def test_normalised_units_link_a_call_that_exact_match_would_orphan():
"""End-to-end: the K-9A2 case that orphaned in the 2026-08-17 01:05Z dump."""
inc = _incident(2.0)
inc["units"] = ["K-9-A-2"]
decision = _run_decision(_ctx(
all_active=[inc], recent=[inc],
call_units=["K-9A2"], is_thin_call=False, call_severity="routine",
))
assert decision["action"] == "link"