Files
server-26/drb-c2-core/app/internal/summarizer.py
T
Logan Cusano d18e4f0743
Build & Deploy / Build & push images (push) Successful in 4m2s
Build & Deploy / Deploy to VM (push) Successful in 1m53s
Build & Deploy / Report a failed deploy (push) Skipped
Make "AI is off" true, and stop the transcript PATCH from destroying calls
config/ai_features was not the switch it was documented to be. Three paths
spent money with it off, and one path read it wrong, so per-system opt-outs
did not opt anything out.

- Correlation in the ingest pipeline tested the raw global flag instead of the
  per-system resolution. With a system opted out, extraction was skipped but
  the no-scenes fallback still correlated the call with empty tags, taking the
  thin/recency path and attaching it to whatever incident was most recent on
  that system. The opt-out did not disable correlation, it disabled good
  correlation and left the worst kind running. (#75)

- Transcript correction ran on every transcribed call gated only by an env var,
  spending Gemini tokens and a Places lookup per proposed location. An
  "STT-only" window was never STT-only and its cost could not be attributed.
  Now behind transcript_correction_enabled. (#76)

- _run_extraction_pipeline and the vocabulary learner, both reachable from
  PATCH /calls/{id}/transcript, checked no flags at all. (#76, #81)

The flag resolver now lives in feature_flags.resolve_flags() rather than as a
local helper in upload.py. Three copies of that logic is how #75 happened.

PATCH /calls/{id}/transcript now refuses with 409 when correlation is off.
That route wipes tags, severity, location, units, embedding and unlinks the
call from every incident before queueing re-extraction. Gating extraction
alone would have made it destructive-only in the standing flags-off
configuration: the call left blank and orphaned forever, with the route still
answering 200. The wipe and the rebuild are one transaction in intent, so it
refuses before the first write.

Also: the summarizer's stale-incident sweep is no longer behind
summaries_enabled. It is pure Firestore with no model call in it, and gating
it meant nothing auto-resolved while AI was off - so every incident stayed
active forever and the candidate set every correlation reads kept growing.

transcript_correction_enabled is documented as NOT a pure cost lever. The
corrector is also the noise gate that sets not_speech; with it off, recogniser
noise reaches extraction as a real transcript, comes back thin, and
auto-attaches. Never open an evaluation window with correction off and
correlation on.

14 tests added covering flag precedence, both pipeline paths, the 409, the
correction gate and the summarizer no-op. Suite: 264 passed.

Refs #75, #76, #81, #45.
2026-08-27 02:49:09 -04:00

173 lines
6.2 KiB
Python

"""
Background incident summary loop.
Runs every SUMMARY_INTERVAL_MINUTES. Two passes per tick:
1. Summary pass — find stale incidents (summary_stale=True) and regenerate summaries.
2. Stale sweep — auto-resolve incidents with no new calls for incident_auto_resolve_minutes.
This is effectively "time since last call" because updated_at is stamped on every
new linked call.
"""
import asyncio
from datetime import datetime, timezone, timedelta
from typing import Optional
from app.internal.logger import logger
from app.internal import firestore as fstore
from app.config import settings
async def summarizer_loop() -> None:
from app.internal.feature_flags import get_flags
interval = settings.summary_interval_minutes * 60
logger.info(f"Summarizer started — interval: {settings.summary_interval_minutes}m")
while True:
await asyncio.sleep(interval)
try:
flags = await get_flags()
if flags["summaries_enabled"]:
await _run_summary_pass()
else:
logger.info("Summaries disabled — skipping summary pass")
# Deliberately outside the flag. Auto-resolving a quiet incident is
# pure Firestore with no model call in it, and gating it behind the
# AI kill switch meant nothing ever auto-resolved in the standing
# flags-off configuration — leaving every incident "active" forever
# and growing the candidate set every correlation reads.
await _resolve_stale_incidents()
except Exception as e:
logger.error(f"Summarizer pass failed: {e}")
async def _run_summary_pass() -> None:
stale = await fstore.collection_list("incidents", status="active", summary_stale=True)
if not stale:
return
logger.info(f"Summarizer: processing {len(stale)} stale incident(s)")
for inc in stale:
await _summarize_incident(inc)
async def _summarize_incident(inc: dict) -> None:
from app.internal.feature_flags import get_flags
incident_id = inc.get("incident_id")
if not incident_id:
return
flags = await get_flags()
if not flags["summaries_enabled"]:
logger.info(f"Summaries disabled — skipping summary for incident {incident_id}")
return
call_ids: list[str] = inc.get("call_ids", [])
if not call_ids:
return
# Fetch transcripts for all calls in this incident
transcripts: list[str] = []
for cid in call_ids:
doc = await fstore.doc_get("calls", cid)
if doc and doc.get("transcript"):
transcripts.append(doc["transcript"])
if not transcripts:
# No transcripts yet — clear stale flag and wait for next pass
await fstore.doc_set("incidents", incident_id, {"summary_stale": False})
return
summary = await asyncio.to_thread(_sync_summarize, inc, transcripts)
now = datetime.now(timezone.utc).isoformat()
updates: dict = {
"summary_stale": False,
"summary_last_run": now,
}
if summary:
updates["summary"] = summary
logger.info(f"Summarizer: updated summary for incident {incident_id}")
else:
logger.warning(f"Summarizer: Gemini returned nothing for incident {incident_id}")
await fstore.doc_set("incidents", incident_id, updates)
async def _resolve_stale_incidents() -> None:
"""Auto-resolve active incidents that have had no new calls for incident_auto_resolve_minutes."""
all_active = await fstore.collection_list("incidents", status="active")
if not all_active:
return
now = datetime.now(timezone.utc)
cutoff = timedelta(minutes=settings.incident_auto_resolve_minutes)
count = 0
for inc in all_active:
incident_id = inc.get("incident_id")
if not incident_id:
continue
try:
updated_dt = datetime.fromisoformat(
str(inc.get("updated_at", "")).replace("Z", "+00:00")
)
if updated_dt.tzinfo is None:
updated_dt = updated_dt.replace(tzinfo=timezone.utc)
idle_minutes = (now - updated_dt).total_seconds() / 60
if idle_minutes > settings.incident_auto_resolve_minutes:
await fstore.doc_set("incidents", incident_id, {
"status": "resolved",
"resolved_at": now.isoformat(),
})
from app.internal.incident_correlator import maybe_resolve_parent
await maybe_resolve_parent(incident_id)
logger.info(
f"Auto-resolved stale incident {incident_id} "
f"(idle {idle_minutes:.0f}m)"
)
count += 1
except Exception as e:
logger.warning(f"Stale sweep error for {incident_id}: {e}")
if count:
logger.info(f"Stale sweep: resolved {count} incident(s)")
def _sync_summarize(inc: dict, transcripts: list[str]) -> Optional[str]:
from app.config import settings
from openai import OpenAI
if not settings.openai_api_key:
return None
inc_type = inc.get("type", "unknown")
location = inc.get("location") or "unknown location"
tg_ids = ", ".join(inc.get("talkgroup_ids", [])) or "unknown"
numbered = "\n".join(f"{i+1}. {t}" for i, t in enumerate(transcripts))
prompt = f"""You are analyzing P25 public safety radio communications for a single active incident.
Incident type: {inc_type}
Location: {location}
Talkgroup(s): {tg_ids}
Transcripts ({len(transcripts)} calls, chronological):
{numbered}
Write a concise factual summary of this incident in 2-4 sentences. Include:
- What happened
- Location (most specific mentioned)
- Units or resources involved if mentioned
- Current status if determinable
Be factual. Do not speculate beyond what the transcripts say. Do not use bullet points."""
try:
client = OpenAI(api_key=settings.openai_api_key)
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": prompt}],
)
return response.choices[0].message.content.strip() or None
except Exception as e:
logger.warning(f"GPT-4o mini summary failed: {e}")
return None