Two independent sources of garbage in the AI pipeline, both visible in the
2026-08-16 correlation dump.
1. Hallucinated transcripts. The Whisper prompt opened with an enumerated run
of ten-codes: 10-4, 10-23, 10-20, 10-97 and so on. Whisper treats prompt
text as preceding transcript, so on noisy or silent audio it continued the
series, emitting transcripts that count upward from 10-4 to 10-99. The
existing no_speech_prob filter could not catch these: the model is highly
confident in text it invented by continuing a pattern.
The prompt no longer contains a series to extend, and _is_degenerate()
rejects the three shapes this failure takes: ascending ten-code runs, one
phrase looping, and near-identical segments across a whole recording.
Verified against 13 transcripts from production: all four known
hallucinations rejected, all nine real ones kept, including terse traffic
containing legitimate codes.
2. Duplicate recordings. node-002 and node-PI-2 both cover TG 9048 and both
uploaded the same transmissions, ~1.1s apart. Nine pairs appeared in one
dump. Each was transcribed, billed and correlated twice, and the resulting
incident listed two units where there was one.
Canonical selection is by earliest started_at, tie-broken on call_id, NOT
by upload order: upload order varies with encode time and network latency,
so it would make the authoritative recording non-deterministic. Call
documents are created from MQTT call_start before uploads arrive, so both
nodes independently reach the same verdict. The loser keeps its audio (it
may be the cleaner capture) but is excluded from STT, correlation, the
re-correlation sweep and the orphan debug view.
Also fixes _sync_transcribe returning a bare None when OPENAI_API_KEY is
missing, where the caller unpacks two values. A missing key surfaced as a
misleading "Transcription failed" instead of the real warning.
Adds tests/test_dedup.py (15 cases). dedup.py reaches Firestore through an
injected callable so it stays importable without firebase-admin present.
_call_fits_incident now returns (bool, signal_str) so each correlation
decision records exactly what evidence fired: unit_overlap, vehicle_overlap,
location_proximity, time_fallback, tactical_default, or the corresponding
false-return variants (unit_loc_conflict, content_divergence, etc.).
- corr_fit_signal and corr_matched_units written to call docs for
fast/single and fast/disambig paths
- Admin debug endpoint exposes the new fields in calls_detail
- Orphan section adds orphans_by_talkgroup summary (count, no-type count,
sweep-exhausted count per TGID) and raises orphan limit 100 → 250
- Admin page shows corr_path and fit_signal distribution panels above raw
JSON; time_fallback highlighted in yellow as a diagnostic marker
No correlation logic changed — diagnostic data only.
Phonetic run threshold 5 → 12: a plate spellout ("Foxtrot Alpha Uniform Lima
Kilo...") produces 6–8 consecutive phonetic words, triggering false positives
and blocking intelligence extraction on legitimate calls. 12 is safely above
any real spellout (~8 max) while still catching the full-alphabet hallucination
(26 words). Also writes skip_reason="garbage_transcript" to the call doc and
surfaces it in the admin correlation debug endpoint.
Logan Cusano