Scene over-splitting: one radio conversation becomes two incidents #5
Closed
opened 2026-08-19 22:06:53 -04:00 by logan
·
5 comments
No Branch/Tag Specified
Labels
Clear labels
area/ai-pipeline
area/auth
area/billing
area/correlation
area/discord
area/frontend
area/governance
area/growth
area/infra-deploy
area/legal
area/map
area/mqtt
area/node-ota
area/osint
area/recording-audio
area/sdr-radio
area/transcription
board
board:agenda
minutes:draft
minutes:final
role:ceo
role:cfo
role:cio
role:ciso
role:cmo
role:coo
role:cto
Scene/entity extraction, summarizer, AI providers, embeddings
AuthN/Z, roles, tenancy/org scoping, API keys, credentials
Plans, entitlements, AI spend accounting and caps
Incident correlation, linking, lifecycle, severity, corr_debug
Discord bot, radio bots, voice streams, bot tokens, webhooks
Web UI: Next.js app and the edge-node local dashboard
Board process, minutes, unattended runner, agent tooling
Go-to-market, outreach, cold email, discovery conversations
CI/CD, VM, registry, Firestore rules/indexes deploy, Terraform/Caddy
ToS, retention, licences/DAAs, CAN-SPAM, E&O, name suppression
Live map: pins, paths, tiles, legend, geocoding
Node-C2 MQTT, checkins, commands, node sweeper, dynsec
Node install, enrollment, OTA updater, image skew
External feeds: cameras, 511, ADS-B, AIS, outages, and their licences
Call capture, boundaries, PulseAudio, ring buffer, trimming, attribution
OP25 decode, SDR hardware/tuning, conventional, second SDR, mesh radio
STT, transcript correction, place verification, name accuracy
Board process: minutes and board-filed items
Needs board scoping at the next sitting; CEO removes the label once scoped
One exec pre-read, closed by CEO
CEO ruling; open until enacted
Filed by the CEO
Filed by the CFO
Filed by the CIO
Filed by the CISO
Filed by the CMO
Filed by the COO
Filed by the CTO
kind
behavior-bug
Wrong product behavior; the instrument reads it correctly
kind
measurement-bug
Corrupts the recording/instrument (debug data, evals, health checks), not behavior
priority
p0
Drop everything: prod broken, live exposure, or owner-ruled P0
priority
p1
Next up: blocks a gate, a customer, or core incident quality
priority
p2
Normal: real defect or planned work, schedule in turn
priority
p3
Low: minor, cosmetic, or someday
status
blocked
Waiting on an external party, spend, or another issue
status
in-progress
Branch or PR open against it
status
needs-decision
Waiting on an owner or board ruling before work
status
needs-info
Root cause or repro unknown; needs a reading first
type
bug
Something built is wrong: code, data, or a measurement
type
chore
Maintenance, refactor, tests, pinning, cleanup: no user-visible change
type
decision
A ruling to make or record: board minutes, owner calls, legal reviews
type
docs
Docs, runbooks, CLAUDE.md, skills, SOPs
type
feature
New capability or an extension of an existing one
type
security
Security, privacy, credential or tenant-isolation exposure
type
tracking
Tracker, checklist, initiative or punch-list spanning several changes
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: logan/server-26#5
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
upload.pycorrelates each extracted scene independently by design, so a call whose transcript yields two scenes can open two separate incidents.Observed 2026-08-19 00:49Z: two calls seconds apart,
10 Parker Streetand10-Parker, apartment 12, both decidedaction=new. Same EMS job, two incidents — each was judged before the other existed, so neither could match the other.Probably a prompt-tuning fix in
intelligence.py(scene extraction granularity) rather than a correlator change, but confirm which layer is over-splitting before touching either. Splitting a real multi-scene call is correct behaviour; the bug is splitting one scene into two.Also tracked in
DEFERRED.md.New evidence from a rules-only-period debug_correlation() pull (100 incidents / 2322 orphans, 2026-08-18 16:47Z - 2026-08-20 02:22Z, all on the same talkgroup, all pre the 08-19 00:49Z LLM boundary): this isn't only the single-call-two-scenes case already described above. Separate calls, seconds to a couple minutes apart on the same TGID, describing the same scene, are opening separate incidents because each is judged before the other exists (same root cause, different trigger).
Strongest examples (TG = talkgroup_ids, dt = gap between the two anchor calls):
0213dbcdcall354e5dc3@ 22:34:37 - "I have train 1777 going into 1-2-5 ... track 4 with an EDP and car number 9-2-9-8" vs incident0429a9be("Edp Report at track 4") call7e666a08@ 22:35:14 (dt=37s) - "11-Victor, can you respond to track 4 for a report of EDP on board train 1777 ... car 9298". Same train, same car, same track, 37 seconds apart, two incidents.917f3313("Other", loc="1 Brookville Avenue, 3rd floor") callfcd9c7bd@ 22:10:00 - "What's the address? 1 Brookville Avenue, 3rd floor" vs incident7a79b0d4("Armed Subject") call31754de4@ 22:10:53 (dt=53s, actually overlaps917f3313's own 22:09:18-22:10:00 span) - "The female with the knife is possibly in the kitchen. 10-11 is locked in her room." Same address, same minute, two incidents.cc4f57cf("Theft Report at South Main Street") call7a7ad18e@ 20:44:55 - "any units in the area of Marshalls ... shelf lifter";73ea2660("Suspect Description at in front of Marshalls") callbec391ca@ 20:47:00 (dt=125s from #1) - "red shirt, a gray sweater ... long hair, wearing blue jeans";9357cf81("Shoplifting") callecdecbac@ 20:55:13 - "the merchandise was recovered ... pursue a charge against the individual." Same store, same TG, one continuous shoplifting call chopped into 3 incidents.9e1d8103("Subject Reported", loc=Pleasantville) calld7400576@ 02:52:11 - "Newcastle PD is advising that subject boarded a train. Subject lives in Pleasantville ... departing" vs incident56c38f23("Police", loc=Pleasantville) calld4812418@ 02:53:00 (dt=49s) - "That's all at Newcastle. PD advised. Subject boarded a train. Lives in Pleasantville. Departing train at Pleasantville." Near-verbatim repeat of the same dispatch, split in two.All four are
corr_path=newon both sides (each call became its own incident anchor) rather than the two-scenes-in-one-call pattern in the issue description, so whatever fix lands for this should also cover cross-call near-duplicates on the same TG within a short window, not just intra-call scene splitting.The failure mode appears to have inverted. In the 2026-08-19 dump there is no over-splitting at all — every one of the 6 incidents has exactly one
newcall. What there is instead is heavy over-merging: 4 of 6 incidents are junk chains, one of them 68 calls over 4h09m covering 13 distinct events (now server-26#22).Before doing prompt-tuning work on scene granularity, re-verify that the "10 Parker Street" / "10-Parker, apartment 12" case still reproduces. Tuning extraction to split less while server-26#22 is merging everything would compound the problem.
One mechanism from the review does still point at real same-event splitting and is worth keeping under this issue: the sweep retry budget can expire before the incident it should have matched exists, so a call that arrives slightly early opens its own incident. That is a timing bug rather than a prompt problem, and it would produce exactly the Parker Street symptom.
Full analysis:
CORRELATION_REVIEW_0820.md(project root, not in git).Measured, 2026-08-26 — over-splitting is confirmed and the system has over-corrected
No new AI window this run (morning slot). These numbers come from analysing the two saved correlation dumps that had never been compared:
corr_0819.json(2026-08-20 02:09Z) and10-10pm_8-23.json(2026-08-24 02:06Z).corr_path=newfast/thinThe correlator did not get better between these two windows, it swung to the opposite failure. 08-19 was severe over-merging — one incident absorbed 68 calls. 08-23 is severe over-splitting: two thirds of every "incident" on the board is a single call, and the median incident is one call, which is not an incident at all.
Where the split is being decided
corr_fit_signalisnoneon 50 of 78 calls (64%) — the rule layer finds no signal whatsoever on two thirds of traffic, so almost everything arrives at the tiebreak with nothing to go on.corr_consensus: tiebreak 38, agreed 29, rules_only 9, none 2.corr_llm_action: orphan 31, link 7, none 40.The LLM tier is not passive — it decides 38 of 78 (49%) — and when it decides, it votes against linking better than 4 to 1. That is the proximate mechanism of the over-split. This also updates #35, which recorded the LLM tier as deciding only 3.5% of links; that is stale, it is now roughly half.
Orphans are not sweep-exhausted — they never reached correlation
Of the 23 orphans, 18 are
transcript_too_shortand 1 isgarbage_transcript. Only 3 carry anycorr_sweep_countat all. The orphan pool is gated at the transcript stage, not lost by the sweep. Relevant to #27, which assumes the sweep budget is what strands them.The map, which is the product
45 of 50 incidents have no
location_coords_source. The 5 that do are unverified free-text phrases from radio traffic ("behind Champa Colas", "South High, in front of Scarborough Church"), not resolved addresses. In the 08-19 dump it was 0 of 6. 90%+ of incidents cannot be plotted, which is the thing being sold. Refs #28, #37, #6.Caveat
Different windows, and the 08-19 dump hit its incident-window cap (
incidents_window_exhausted: true) while 08-23 did not, so the incident counts are not strictly like-for-like. The size distribution andcorr_pathshares are, and those are where the swing shows.Static review the same day (#74, #75, #80) finds the sweep path still leaning over-merge, which is consistent: the main path over-splits, the sweep path over-merges. Fixing one direction without measuring the other is how this got here.
Refs #35, #27, #28, #74, #75, #80.
Measured against 3 real dumps (corr_dump_9-7_0437am, corr_dump_9-7_pm, corr_dump_9-12) — the exact mechanism #5 describes does not reproduce
Scripted a scan for the literal pattern: one
call_idappearing incalls_detailof 2+ different incidents. Found 10 hits total (4 / 2 / 4 across the three windows, out of 173/173/181 linked calls each — ~2%).Every single one of the 10 has byte-identical extracted fields (tags, location, units, severity) between its two incident appearances. Example (
corr_dump_9-7_0437am), transcript: "6-David, that's going to be track 4, intox male in his 40s, track 4, Mt. Vernon East. Affirmative. 10-4." — appears in incident3254fd80("Intoxicated Subject at track 4, Mt. Vernon East") ANDfe04dd3a("Police — MTA PD Districts 6/7/11 - Police Dispatch"), both withtags=['intoxicated-subject'] location='track 4, Mt. Vernon East' units=['6-David']— this is the exact DEFERRED.md-cited 9-6 case, still occurring 9-7, now 2 incidents instead of 3.If
intelligence.pyhad actually split this transcript into 2 differently-described scenes, the two incidents would carry different location/tags/units (each scene extracts its own content). They don't — anywhere, in any of the 10 hits. That rules out the mechanism #5's title and body describe (one call → 2+ scenes → 2+ incidents). Zero genuine intra-call multi-scene over-split cases found.EXTRACTION_PROMPT check: current
intelligence.pySCENE DETECTION block still matches DEFERRED.md's description verbatim — default ONE scene, explicit "when unsure, one scene", not-a-new-scene list intact. No drift since the 2026-08-25 tightening.What's actually happening instead: the same single-scene call is being attached to two separate incident records with identical data — a correlator-side duplicate-linking bug, not a scene-extraction bug.
corr_path/corr_consensuson the two appearances are usually identical too (e.g. bothnew/tiebreak, or bothfast/disambig/agreed), consistent with the same call being run through correlation twice and each pass landing on (or creating) a different incident target without removing the call from the other. This produces the same downstream symptom (double-counted call, inflated incident count) DEFERRED.md attributes to #5, but the root cause is different and lives inincident_correlator.py's linking/sweep path, notintelligence.py.Recommendation: close #5 as not reproducing per its stated mechanism — the 2026-08-25 prompt tightening appears to have worked for genuine intra-call scene over-splitting (0/10 candidates confirm it across 3 windows). Open a fresh issue for "same call_id linked into 2 incident records with identical extracted data" (10 instances, 3 windows, evidence above) — that one needs an engineer reading the sweep/re-correlation call_ids append path in
incident_correlator.py, not prompt work. Recommend linking it from the DEFERRED.md #5 row so the row's hypothesis gets corrected rather than left pointing at a mechanism that isn't the live bug.(Investigation only — no code changed.)
Closing per the investigation above: #5's stated mechanism (one call, transcript splits into 2+ scenes, each opens an incident) does not reproduce in 3 live dumps (0/10 candidate hits show differing extracted content between appearances) and the EXTRACTION_PROMPT has not drifted from the 2026-08-25 tightening. The real bug producing the same symptom (a call double-counted across incidents) is a correlator-side duplicate-linking issue, not scene extraction — filed as #131, which supersedes this issue. DEFERRED.md's #5 row updated to point at #131.