Let severity, not incident_type, decide what becomes an incident
The 2026-08-16 correlation dump showed two failures that looked unrelated and were the same bug. TG 9048 held one incident of 28 calls spanning 49 minutes -- a prisoner transport, a drone retrieval, a records lookup and a canvass, glued together -- while 32 other calls on that same channel stayed permanently orphaned. Creating an incident required a concrete incident_type. Nothing on a transit police channel produced one: the extraction prompt said to prefer "other" when uncertain, extraction then collapsed "other" to None, and the tag-based fallback had no tags to work with because administrative traffic carries none. So the channel could never open a SECOND incident. Every later call funnelled into whichever incident happened to exist first, and every call too substantial for the thin path had nowhere to go at all. The two symptoms were the same missing value seen from opposite ends. Severity now decides incident-worthiness. It is a better fit for the question being asked -- "is this a real event?" -- than a service label ever was, and unlike incident_type it is always present. The prompt defines four levels with no escape hatch (routine/minor/moderate/major, "unknown" is gone) and calls skipped for a too-short transcript are still recorded as routine, because downstream code reads a missing severity as "not processed yet" rather than "nothing happened". Anything above routine, or carrying any extracted content, opens an incident under the neutral "other" type. "other" is also kept as a real classification now -- rail operations and public works genuinely are not police, fire or EMS. Separately, thin calls no longer refresh updated_at; they write last_thin_at. updated_at drives every recency gate in the fast path, so each "10-4" was resetting the idle clock on whatever it attached to, keeping that incident inside the gate for as long as anyone kept acknowledging. An incident now ages from its last substantive call. This is what made the 49-minute incident possible even once buckets existed, so it is fixed independently rather than being left to the gate change. The re-correlation sweep also now honours skip_reason. /upload has always refused to correlate garbage and too-short transcripts, but the sweep did not apply the same filter, so those fragments came back minutes later through the thin path and attached to whatever was most recent -- a second, quieter route into the same over-merge. Adds tests/test_correlator_gate.py (15 cases), the first tests against incident_correlator.py in its 1,517-line history. tests/conftest.py stubs firebase-admin only when it is genuinely absent, so the container's real SDK is never shadowed; this is what makes the correlator importable in the dev venv. That stub also made test_mqtt_handler and test_node_sweeper collectable for the first time, revealing 10 pre-existing failures in them -- test-vs-code drift, untouched here and catalogued in DEFERRED.md. No new environment variables, so CI deploys this without an ansible run.
This commit is contained in:
@@ -1,2 +1,61 @@
|
||||
# All C2 core settings have defaults — no env setup needed.
|
||||
# Add any shared fixtures here if required in the future.
|
||||
#
|
||||
# firebase-admin and google-cloud-firestore are runtime-only dependencies: they
|
||||
# are installed in the container but not in the local dev venv, and
|
||||
# app/internal/firestore.py calls _init_firebase() at import time. Without the
|
||||
# stubs below, importing ANY module that reaches Firestore fails at collection
|
||||
# time, which is why test_mqtt_handler and test_node_sweeper could not be run
|
||||
# outside the container.
|
||||
#
|
||||
# The stubs are installed only when the real packages are absent, so the
|
||||
# container's real SDK is never shadowed.
|
||||
import sys
|
||||
from types import ModuleType
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
try: # pragma: no cover - exercised only by which packages are installed
|
||||
import firebase_admin # noqa: F401
|
||||
except ModuleNotFoundError:
|
||||
_firebase = ModuleType("firebase_admin")
|
||||
# Falsy so _init_firebase() takes the initialize_app() branch rather than the
|
||||
# already-initialised branch, which is itself broken (see DEFERRED.md).
|
||||
_firebase._apps = {}
|
||||
_firebase.initialize_app = MagicMock()
|
||||
_firebase.credentials = MagicMock()
|
||||
_firebase.firestore = MagicMock()
|
||||
|
||||
_credentials = ModuleType("firebase_admin.credentials")
|
||||
_credentials.Certificate = MagicMock()
|
||||
_credentials.ApplicationDefault = MagicMock()
|
||||
|
||||
_fs = ModuleType("firebase_admin.firestore")
|
||||
_fs.client = MagicMock()
|
||||
# A distinct sentinel rather than a MagicMock: production code writes this
|
||||
# into dicts that tests compare against, and a MagicMock compares unequal
|
||||
# to itself across attribute accesses.
|
||||
_fs.SERVER_TIMESTAMP = "__SERVER_TIMESTAMP__"
|
||||
|
||||
_auth = ModuleType("firebase_admin.auth")
|
||||
_auth.verify_id_token = MagicMock()
|
||||
_auth.set_custom_user_claims = MagicMock()
|
||||
_auth.get_user_by_email = MagicMock()
|
||||
_auth.get_user = MagicMock()
|
||||
|
||||
_firebase.auth = _auth
|
||||
_firebase.credentials = _credentials
|
||||
_firebase.firestore = _fs
|
||||
|
||||
sys.modules["firebase_admin"] = _firebase
|
||||
sys.modules["firebase_admin.credentials"] = _credentials
|
||||
sys.modules["firebase_admin.firestore"] = _fs
|
||||
sys.modules["firebase_admin.auth"] = _auth
|
||||
|
||||
try: # pragma: no cover
|
||||
from google.cloud.firestore_v1.base_query import FieldFilter # noqa: F401
|
||||
except ModuleNotFoundError:
|
||||
for _name in (
|
||||
"google", "google.cloud", "google.cloud.firestore_v1",
|
||||
"google.cloud.firestore_v1.base_query",
|
||||
):
|
||||
sys.modules.setdefault(_name, ModuleType(_name))
|
||||
sys.modules["google.cloud.firestore_v1.base_query"].FieldFilter = MagicMock()
|
||||
|
||||
@@ -0,0 +1,148 @@
|
||||
"""
|
||||
Unit tests for the incident-creation gate and the thin-call activity rule.
|
||||
|
||||
Both behaviours come from the 2026-08-16 correlation dump, where TG 9048
|
||||
produced one 28-call / 49-minute incident alongside 32 permanent orphans:
|
||||
|
||||
* Requiring a concrete incident_type to create an incident meant a channel
|
||||
whose traffic never classifies could never open a second incident, so every
|
||||
later call funnelled into whichever incident existed first.
|
||||
* Thin ("10-4") calls refreshed updated_at, which kept that incident
|
||||
permanently inside the fast-path recency gate.
|
||||
|
||||
_run_decision is pure — it reads only the context dict — so these cases need no
|
||||
Firestore. _update_incident writes, so its test patches fstore.
|
||||
"""
|
||||
import pytest
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from unittest.mock import AsyncMock, patch
|
||||
from app.internal.incident_correlator import _run_decision, _update_incident
|
||||
|
||||
NOW = datetime(2026, 8, 16, 21, 0, 0, tzinfo=timezone.utc)
|
||||
|
||||
|
||||
def _ctx(**overrides) -> dict:
|
||||
"""Context with no active incidents, so the decision reaches the creation gate."""
|
||||
base = {
|
||||
"call_id": "call-1",
|
||||
"all_active": [],
|
||||
"recent": [],
|
||||
"call_doc": {},
|
||||
"call_embedding": None,
|
||||
"call_units": [],
|
||||
"call_vehicles": [],
|
||||
"call_cleared": [],
|
||||
"call_severity": "routine",
|
||||
"coords": None,
|
||||
"is_thin_call": True,
|
||||
"now": NOW,
|
||||
"system_id": "sys-1",
|
||||
"talkgroup_id": 9048,
|
||||
"talkgroup_name": "MTA PD Districts 6/7/11 - Police Dispatch",
|
||||
"tags": [],
|
||||
"incident_type": None,
|
||||
"location": None,
|
||||
"location_coords": None,
|
||||
"reassignment": False,
|
||||
"create_if_new": True,
|
||||
}
|
||||
base.update(overrides)
|
||||
return base
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Creation gate — severity decides incident-worthiness, not incident_type
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_routine_status_traffic_stays_orphaned():
|
||||
"""A content-free acknowledgement must not open an incident of its own."""
|
||||
assert _run_decision(_ctx())["action"] == "orphan"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("severity", ["minor", "moderate", "major"])
|
||||
def test_any_real_severity_opens_an_untyped_incident(severity):
|
||||
decision = _run_decision(_ctx(call_severity=severity))
|
||||
assert decision["action"] == "new"
|
||||
assert decision["incident_type"] == "other"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("field,value", [
|
||||
("call_units", ["6 Adam"]),
|
||||
("call_vehicles", ["RMP 22146"]),
|
||||
("coords", {"lat": 41.0, "lng": -73.8}),
|
||||
("location", "District 6"),
|
||||
("tags", ["prisoner-transport"]),
|
||||
])
|
||||
def test_concrete_content_opens_an_untyped_incident(field, value):
|
||||
"""Routine severity is overridden by anything the extractor actually found."""
|
||||
decision = _run_decision(_ctx(**{field: value}))
|
||||
assert decision["action"] == "new"
|
||||
assert decision["incident_type"] == "other"
|
||||
|
||||
|
||||
def test_explicit_type_is_never_downgraded_to_other():
|
||||
decision = _run_decision(_ctx(incident_type="police", call_severity="moderate"))
|
||||
assert decision["action"] == "new"
|
||||
assert decision["incident_type"] == "police"
|
||||
|
||||
|
||||
def test_other_survives_extraction_and_creates_an_incident():
|
||||
""""other" is a real classification now, not a synonym for unclassifiable."""
|
||||
decision = _run_decision(_ctx(incident_type="other"))
|
||||
assert decision["action"] == "new"
|
||||
assert decision["incident_type"] == "other"
|
||||
|
||||
|
||||
def test_sweep_never_creates_incidents():
|
||||
"""The re-correlation sweep passes create_if_new=False — it may only link."""
|
||||
decision = _run_decision(_ctx(call_severity="major", create_if_new=False))
|
||||
assert decision["action"] == "orphan"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Thin calls attach for context but do not count as incident activity
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _incident(idle_minutes: float) -> dict:
|
||||
updated = NOW - timedelta(minutes=idle_minutes)
|
||||
return {
|
||||
"incident_id": "inc-1",
|
||||
"system_ids": ["sys-1"],
|
||||
"talkgroup_ids": ["9048"],
|
||||
"updated_at": updated.isoformat(),
|
||||
"started_at": updated.isoformat(),
|
||||
"status": "active",
|
||||
}
|
||||
|
||||
|
||||
def test_thin_call_links_to_the_active_incident_on_its_talkgroup():
|
||||
inc = _incident(0.2)
|
||||
decision = _run_decision(_ctx(all_active=[inc], recent=[inc]))
|
||||
assert decision["action"] == "link"
|
||||
assert decision["corr_debug"]["corr_path"] == "fast/thin"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_thin_link_does_not_refresh_updated_at():
|
||||
with patch("app.internal.incident_correlator.fstore") as mock_fstore:
|
||||
mock_fstore.doc_set = AsyncMock()
|
||||
await _update_incident(
|
||||
_incident(5), "call-1", 9048, "sys-1", [], None, None, [], [], None, NOW,
|
||||
refresh_activity=False,
|
||||
)
|
||||
updates = mock_fstore.doc_set.await_args.args[2]
|
||||
assert "updated_at" not in updates, "a '10-4' must not reset the incident idle clock"
|
||||
assert updates["last_thin_at"] == NOW.isoformat()
|
||||
assert updates["summary_stale"] is True, "the call still belongs in the summary"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_substantive_link_does_refresh_updated_at():
|
||||
with patch("app.internal.incident_correlator.fstore") as mock_fstore:
|
||||
mock_fstore.doc_set = AsyncMock()
|
||||
await _update_incident(
|
||||
_incident(5), "call-1", 9048, "sys-1", [], None, None, ["6 Adam"], [], None, NOW,
|
||||
)
|
||||
updates = mock_fstore.doc_set.await_args.args[2]
|
||||
assert updates["updated_at"] == NOW.isoformat()
|
||||
assert "last_thin_at" not in updates
|
||||
Reference in New Issue
Block a user