upload.py correlates each extracted scene independently by design, so a call whose transcript yields two scenes can open two separate incidents.
Observed 2026-08-19 00:49Z: two calls seconds apart, 10 Parker Street and 10-Parker, apartment 12, both decided action=new. Same EMS job, two incidents — each was judged before the other existed, so neither could match the other.
Probably a prompt-tuning fix in intelligence.py (scene extraction granularity) rather than a correlator change, but confirm which layer is over-splitting before touching either. Splitting a real multi-scene call is correct behaviour; the bug is splitting one scene into two.
Also tracked in DEFERRED.md.
`upload.py` correlates each extracted scene independently by design, so a call whose transcript yields two scenes can open two separate incidents.
**Observed 2026-08-19 00:49Z:** two calls seconds apart, `10 Parker Street` and `10-Parker, apartment 12`, both decided `action=new`. Same EMS job, two incidents — each was judged before the other existed, so neither could match the other.
Probably a prompt-tuning fix in `intelligence.py` (scene extraction granularity) rather than a correlator change, but confirm which layer is over-splitting before touching either. Splitting a real multi-scene call is correct behaviour; the bug is splitting one scene into two.
Also tracked in `DEFERRED.md`.
New evidence from a rules-only-period debug_correlation() pull (100 incidents / 2322 orphans, 2026-08-18 16:47Z - 2026-08-20 02:22Z, all on the same talkgroup, all pre the 08-19 00:49Z LLM boundary): this isn't only the single-call-two-scenes case already described above. Separate calls, seconds to a couple minutes apart on the same TGID, describing the same scene, are opening separate incidents because each is judged before the other exists (same root cause, different trigger).
Strongest examples (TG = talkgroup_ids, dt = gap between the two anchor calls):
EDP on train 1777, car 9298, track 4 (TG 9048): incident 0213dbcd call 354e5dc3 @ 22:34:37 - "I have train 1777 going into 1-2-5 ... track 4 with an EDP and car number 9-2-9-8" vs incident 0429a9be ("Edp Report at track 4") call 7e666a08 @ 22:35:14 (dt=37s) - "11-Victor, can you respond to track 4 for a report of EDP on board train 1777 ... car 9298". Same train, same car, same track, 37 seconds apart, two incidents.
Armed subject w/ knife, 1 Brookville Ave 3rd floor (TG 9600): incident 917f3313 ("Other", loc="1 Brookville Avenue, 3rd floor") call fcd9c7bd @ 22:10:00 - "What's the address? 1 Brookville Avenue, 3rd floor" vs incident 7a79b0d4 ("Armed Subject") call 31754de4 @ 22:10:53 (dt=53s, actually overlaps 917f3313's own 22:09:18-22:10:00 span) - "The female with the knife is possibly in the kitchen. 10-11 is locked in her room." Same address, same minute, two incidents.
Shoplifter at Marshalls (TG 9211), three separate incidents for what reads as one event over ~11 minutes: cc4f57cf ("Theft Report at South Main Street") call 7a7ad18e @ 20:44:55 - "any units in the area of Marshalls ... shelf lifter"; 73ea2660 ("Suspect Description at in front of Marshalls") call bec391ca @ 20:47:00 (dt=125s from #1) - "red shirt, a gray sweater ... long hair, wearing blue jeans"; 9357cf81 ("Shoplifting") call ecdecbac @ 20:55:13 - "the merchandise was recovered ... pursue a charge against the individual." Same store, same TG, one continuous shoplifting call chopped into 3 incidents.
Subject boarding train to Pleasantville (TG 9048): incident 9e1d8103 ("Subject Reported", loc=Pleasantville) call d7400576 @ 02:52:11 - "Newcastle PD is advising that subject boarded a train. Subject lives in Pleasantville ... departing" vs incident 56c38f23 ("Police", loc=Pleasantville) call d4812418 @ 02:53:00 (dt=49s) - "That's all at Newcastle. PD advised. Subject boarded a train. Lives in Pleasantville. Departing train at Pleasantville." Near-verbatim repeat of the same dispatch, split in two.
All four are corr_path=new on both sides (each call became its own incident anchor) rather than the two-scenes-in-one-call pattern in the issue description, so whatever fix lands for this should also cover cross-call near-duplicates on the same TG within a short window, not just intra-call scene splitting.
New evidence from a rules-only-period debug_correlation() pull (100 incidents / 2322 orphans, 2026-08-18 16:47Z - 2026-08-20 02:22Z, all on the same talkgroup, all pre the 08-19 00:49Z LLM boundary): this isn't only the single-call-two-scenes case already described above. Separate *calls*, seconds to a couple minutes apart on the same TGID, describing the same scene, are opening separate incidents because each is judged before the other exists (same root cause, different trigger).
Strongest examples (TG = talkgroup_ids, dt = gap between the two anchor calls):
1. **EDP on train 1777, car 9298, track 4** (TG 9048): incident `0213dbcd` call `354e5dc3` @ 22:34:37 - "I have train 1777 going into 1-2-5 ... track 4 with an EDP and car number 9-2-9-8" vs incident `0429a9be` ("Edp Report at track 4") call `7e666a08` @ 22:35:14 (dt=37s) - "11-Victor, can you respond to track 4 for a report of EDP on board train 1777 ... car 9298". Same train, same car, same track, 37 seconds apart, two incidents.
2. **Armed subject w/ knife, 1 Brookville Ave 3rd floor** (TG 9600): incident `917f3313` ("Other", loc="1 Brookville Avenue, 3rd floor") call `fcd9c7bd` @ 22:10:00 - "What's the address? 1 Brookville Avenue, 3rd floor" vs incident `7a79b0d4` ("Armed Subject") call `31754de4` @ 22:10:53 (dt=53s, actually overlaps `917f3313`'s own 22:09:18-22:10:00 span) - "The female with the knife is possibly in the kitchen. 10-11 is locked in her room." Same address, same minute, two incidents.
3. **Shoplifter at Marshalls** (TG 9211), three separate incidents for what reads as one event over ~11 minutes: `cc4f57cf` ("Theft Report at South Main Street") call `7a7ad18e` @ 20:44:55 - "any units in the area of Marshalls ... shelf lifter"; `73ea2660` ("Suspect Description at in front of Marshalls") call `bec391ca` @ 20:47:00 (dt=125s from #1) - "red shirt, a gray sweater ... long hair, wearing blue jeans"; `9357cf81` ("Shoplifting") call `ecdecbac` @ 20:55:13 - "the merchandise was recovered ... pursue a charge against the individual." Same store, same TG, one continuous shoplifting call chopped into 3 incidents.
4. **Subject boarding train to Pleasantville** (TG 9048): incident `9e1d8103` ("Subject Reported", loc=Pleasantville) call `d7400576` @ 02:52:11 - "Newcastle PD is advising that subject boarded a train. Subject lives in Pleasantville ... departing" vs incident `56c38f23` ("Police", loc=Pleasantville) call `d4812418` @ 02:53:00 (dt=49s) - "That's all at Newcastle. PD advised. Subject boarded a train. Lives in Pleasantville. Departing train at Pleasantville." Near-verbatim repeat of the same dispatch, split in two.
All four are `corr_path=new` on both sides (each call became its own incident anchor) rather than the two-scenes-in-one-call pattern in the issue description, so whatever fix lands for this should also cover cross-call near-duplicates on the same TG within a short window, not just intra-call scene splitting.
The failure mode appears to have inverted. In the 2026-08-19 dump there is no over-splitting at all — every one of the 6 incidents has exactly one new call. What there is instead is heavy over-merging: 4 of 6 incidents are junk chains, one of them 68 calls over 4h09m covering 13 distinct events (now server-26#22).
Before doing prompt-tuning work on scene granularity, re-verify that the "10 Parker Street" / "10-Parker, apartment 12" case still reproduces. Tuning extraction to split less while server-26#22 is merging everything would compound the problem.
One mechanism from the review does still point at real same-event splitting and is worth keeping under this issue: the sweep retry budget can expire before the incident it should have matched exists, so a call that arrives slightly early opens its own incident. That is a timing bug rather than a prompt problem, and it would produce exactly the Parker Street symptom.
Full analysis: CORRELATION_REVIEW_0820.md (project root, not in git).
**The failure mode appears to have inverted.** In the 2026-08-19 dump there is no over-splitting at all — every one of the 6 incidents has exactly one `new` call. What there is instead is heavy over-*merging*: 4 of 6 incidents are junk chains, one of them 68 calls over 4h09m covering 13 distinct events (now server-26#22).
Before doing prompt-tuning work on scene granularity, re-verify that the "10 Parker Street" / "10-Parker, apartment 12" case still reproduces. Tuning extraction to split *less* while server-26#22 is merging everything would compound the problem.
One mechanism from the review does still point at real same-event splitting and is worth keeping under this issue: **the sweep retry budget can expire before the incident it should have matched exists**, so a call that arrives slightly early opens its own incident. That is a timing bug rather than a prompt problem, and it would produce exactly the Parker Street symptom.
Full analysis: `CORRELATION_REVIEW_0820.md` (project root, not in git).
Measured, 2026-08-26 — over-splitting is confirmed and the system has over-corrected
No new AI window this run (morning slot). These numbers come from analysing the two saved correlation dumps that had never been compared: corr_0819.json (2026-08-20 02:09Z) and 10-10pm_8-23.json (2026-08-24 02:06Z).
08-19 window
08-23 window
linked calls
133
78
incidents
6
50
median calls / incident
11.5
1
largest incident
68 calls
7 calls
corr_path=new
6 (4.5%)
51 (65%)
fast/thin
58 (44%)
7 (9%)
single-call incidents
0
33 of 50 (66%)
orphans
359
23
The correlator did not get better between these two windows, it swung to the opposite failure. 08-19 was severe over-merging — one incident absorbed 68 calls. 08-23 is severe over-splitting: two thirds of every "incident" on the board is a single call, and the median incident is one call, which is not an incident at all.
Where the split is being decided
corr_fit_signal is none on 50 of 78 calls (64%) — the rule layer finds no signal whatsoever on two thirds of traffic, so almost everything arrives at the tiebreak with nothing to go on.
The LLM tier is not passive — it decides 38 of 78 (49%) — and when it decides, it votes against linking better than 4 to 1. That is the proximate mechanism of the over-split. This also updates #35, which recorded the LLM tier as deciding only 3.5% of links; that is stale, it is now roughly half.
Orphans are not sweep-exhausted — they never reached correlation
Of the 23 orphans, 18 are transcript_too_short and 1 is garbage_transcript. Only 3 carry any corr_sweep_count at all. The orphan pool is gated at the transcript stage, not lost by the sweep. Relevant to #27, which assumes the sweep budget is what strands them.
The map, which is the product
45 of 50 incidents have no location_coords_source. The 5 that do are unverified free-text phrases from radio traffic ("behind Champa Colas", "South High, in front of Scarborough Church"), not resolved addresses. In the 08-19 dump it was 0 of 6. 90%+ of incidents cannot be plotted, which is the thing being sold. Refs #28, #37, #6.
Caveat
Different windows, and the 08-19 dump hit its incident-window cap (incidents_window_exhausted: true) while 08-23 did not, so the incident counts are not strictly like-for-like. The size distribution and corr_path shares are, and those are where the swing shows.
Static review the same day (#74, #75, #80) finds the sweep path still leaning over-merge, which is consistent: the main path over-splits, the sweep path over-merges. Fixing one direction without measuring the other is how this got here.
## Measured, 2026-08-26 — over-splitting is confirmed and the system has over-corrected
No new AI window this run (morning slot). These numbers come from analysing the two saved correlation dumps that had never been compared: `corr_0819.json` (2026-08-20 02:09Z) and `10-10pm_8-23.json` (2026-08-24 02:06Z).
| | 08-19 window | 08-23 window |
|---|---|---|
| linked calls | 133 | 78 |
| incidents | 6 | 50 |
| median calls / incident | 11.5 | **1** |
| largest incident | **68 calls** | 7 calls |
| `corr_path=new` | 6 (4.5%) | **51 (65%)** |
| `fast/thin` | 58 (44%) | 7 (9%) |
| single-call incidents | 0 | **33 of 50 (66%)** |
| orphans | 359 | 23 |
**The correlator did not get better between these two windows, it swung to the opposite failure.** 08-19 was severe over-merging — one incident absorbed 68 calls. 08-23 is severe over-splitting: two thirds of every "incident" on the board is a single call, and the median incident is one call, which is not an incident at all.
### Where the split is being decided
- `corr_fit_signal` is `none` on **50 of 78** calls (64%) — the rule layer finds no signal whatsoever on two thirds of traffic, so almost everything arrives at the tiebreak with nothing to go on.
- `corr_consensus`: tiebreak 38, agreed 29, rules_only 9, none 2.
- `corr_llm_action`: **orphan 31, link 7**, none 40.
The LLM tier is not passive — it decides 38 of 78 (49%) — and when it decides, **it votes against linking better than 4 to 1**. That is the proximate mechanism of the over-split. **This also updates #35**, which recorded the LLM tier as deciding only 3.5% of links; that is stale, it is now roughly half.
### Orphans are not sweep-exhausted — they never reached correlation
Of the 23 orphans, **18 are `transcript_too_short`** and 1 is `garbage_transcript`. Only 3 carry any `corr_sweep_count` at all. The orphan pool is gated at the transcript stage, not lost by the sweep. Relevant to #27, which assumes the sweep budget is what strands them.
### The map, which is the product
**45 of 50 incidents have no `location_coords_source`.** The 5 that do are unverified free-text phrases from radio traffic ("behind Champa Colas", "South High, in front of Scarborough Church"), not resolved addresses. In the 08-19 dump it was 0 of 6. **90%+ of incidents cannot be plotted**, which is the thing being sold. Refs #28, #37, #6.
### Caveat
Different windows, and the 08-19 dump hit its incident-window cap (`incidents_window_exhausted: true`) while 08-23 did not, so the incident counts are not strictly like-for-like. The size distribution and `corr_path` shares are, and those are where the swing shows.
Static review the same day (#74, #75, #80) finds the **sweep** path still leaning over-merge, which is consistent: the main path over-splits, the sweep path over-merges. Fixing one direction without measuring the other is how this got here.
Refs #35, #27, #28, #74, #75, #80.
Measured against 3 real dumps (corr_dump_9-7_0437am, corr_dump_9-7_pm, corr_dump_9-12) — the exact mechanism #5 describes does not reproduce
Scripted a scan for the literal pattern: one call_id appearing in calls_detail of 2+ different incidents. Found 10 hits total (4 / 2 / 4 across the three windows, out of 173/173/181 linked calls each — ~2%).
Every single one of the 10 has byte-identical extracted fields (tags, location, units, severity) between its two incident appearances. Example (corr_dump_9-7_0437am), transcript: "6-David, that's going to be track 4, intox male in his 40s, track 4, Mt. Vernon East. Affirmative. 10-4." — appears in incident 3254fd80 ("Intoxicated Subject at track 4, Mt. Vernon East") AND fe04dd3a ("Police — MTA PD Districts 6/7/11 - Police Dispatch"), both with tags=['intoxicated-subject'] location='track 4, Mt. Vernon East' units=['6-David'] — this is the exact DEFERRED.md-cited 9-6 case, still occurring 9-7, now 2 incidents instead of 3.
If intelligence.py had actually split this transcript into 2 differently-described scenes, the two incidents would carry different location/tags/units (each scene extracts its own content). They don't — anywhere, in any of the 10 hits. That rules out the mechanism #5's title and body describe (one call → 2+ scenes → 2+ incidents). Zero genuine intra-call multi-scene over-split cases found.
EXTRACTION_PROMPT check: current intelligence.py SCENE DETECTION block still matches DEFERRED.md's description verbatim — default ONE scene, explicit "when unsure, one scene", not-a-new-scene list intact. No drift since the 2026-08-25 tightening.
What's actually happening instead: the same single-scene call is being attached to two separate incident records with identical data — a correlator-side duplicate-linking bug, not a scene-extraction bug. corr_path/corr_consensus on the two appearances are usually identical too (e.g. both new/tiebreak, or both fast/disambig/agreed), consistent with the same call being run through correlation twice and each pass landing on (or creating) a different incident target without removing the call from the other. This produces the same downstream symptom (double-counted call, inflated incident count) DEFERRED.md attributes to #5, but the root cause is different and lives in incident_correlator.py's linking/sweep path, not intelligence.py.
Recommendation:close#5 as not reproducing per its stated mechanism — the 2026-08-25 prompt tightening appears to have worked for genuine intra-call scene over-splitting (0/10 candidates confirm it across 3 windows). Open a fresh issue for "same call_id linked into 2 incident records with identical extracted data" (10 instances, 3 windows, evidence above) — that one needs an engineer reading the sweep/re-correlation call_ids append path in incident_correlator.py, not prompt work. Recommend linking it from the DEFERRED.md #5 row so the row's hypothesis gets corrected rather than left pointing at a mechanism that isn't the live bug.
(Investigation only — no code changed.)
## Measured against 3 real dumps (corr_dump_9-7_0437am, corr_dump_9-7_pm, corr_dump_9-12) — the exact mechanism #5 describes does not reproduce
Scripted a scan for the literal pattern: one `call_id` appearing in `calls_detail` of 2+ *different* incidents. Found 10 hits total (4 / 2 / 4 across the three windows, out of 173/173/181 linked calls each — ~2%).
**Every single one of the 10 has byte-identical extracted fields (tags, location, units, severity) between its two incident appearances.** Example (`corr_dump_9-7_0437am`), transcript: *"6-David, that's going to be track 4, intox male in his 40s, track 4, Mt. Vernon East. Affirmative. 10-4."* — appears in incident `3254fd80` ("Intoxicated Subject at track 4, Mt. Vernon East") AND `fe04dd3a` ("Police — MTA PD Districts 6/7/11 - Police Dispatch"), both with `tags=['intoxicated-subject'] location='track 4, Mt. Vernon East' units=['6-David']` — this is the exact DEFERRED.md-cited 9-6 case, still occurring 9-7, now 2 incidents instead of 3.
If `intelligence.py` had actually split this transcript into 2 differently-described scenes, the two incidents would carry *different* location/tags/units (each scene extracts its own content). They don't — anywhere, in any of the 10 hits. That rules out the mechanism #5's title and body describe (one call → 2+ scenes → 2+ incidents). **Zero genuine intra-call multi-scene over-split cases found.**
**EXTRACTION_PROMPT check:** current `intelligence.py` SCENE DETECTION block still matches DEFERRED.md's description verbatim — default ONE scene, explicit "when unsure, one scene", not-a-new-scene list intact. No drift since the 2026-08-25 tightening.
**What's actually happening instead:** the same single-scene call is being attached to *two separate incident records* with identical data — a correlator-side duplicate-linking bug, not a scene-extraction bug. `corr_path`/`corr_consensus` on the two appearances are usually identical too (e.g. both `new`/`tiebreak`, or both `fast/disambig`/`agreed`), consistent with the same call being run through correlation twice and each pass landing on (or creating) a different incident target without removing the call from the other. This produces the same downstream symptom (double-counted call, inflated incident count) DEFERRED.md attributes to #5, but the root cause is different and lives in `incident_correlator.py`'s linking/sweep path, not `intelligence.py`.
**Recommendation:** close #5 as not reproducing per its stated mechanism — the 2026-08-25 prompt tightening appears to have worked for genuine intra-call scene over-splitting (0/10 candidates confirm it across 3 windows). Open a fresh issue for "same call_id linked into 2 incident records with identical extracted data" (10 instances, 3 windows, evidence above) — that one needs an engineer reading the sweep/re-correlation call_ids append path in `incident_correlator.py`, not prompt work. Recommend linking it from the DEFERRED.md #5 row so the row's hypothesis gets corrected rather than left pointing at a mechanism that isn't the live bug.
(Investigation only — no code changed.)
Closing per the investigation above: #5's stated mechanism (one call, transcript splits into 2+ scenes, each opens an incident) does not reproduce in 3 live dumps (0/10 candidate hits show differing extracted content between appearances) and the EXTRACTION_PROMPT has not drifted from the 2026-08-25 tightening. The real bug producing the same symptom (a call double-counted across incidents) is a correlator-side duplicate-linking issue, not scene extraction — filed as #131, which supersedes this issue. DEFERRED.md's #5 row updated to point at #131.
Closing per the investigation above: #5's stated mechanism (one call, transcript splits into 2+ scenes, each opens an incident) does not reproduce in 3 live dumps (0/10 candidate hits show differing extracted content between appearances) and the EXTRACTION_PROMPT has not drifted from the 2026-08-25 tightening. The real bug producing the same symptom (a call double-counted across incidents) is a correlator-side duplicate-linking issue, not scene extraction — filed as #131, which supersedes this issue. DEFERRED.md's #5 row updated to point at #131.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
upload.pycorrelates each extracted scene independently by design, so a call whose transcript yields two scenes can open two separate incidents.Observed 2026-08-19 00:49Z: two calls seconds apart,
10 Parker Streetand10-Parker, apartment 12, both decidedaction=new. Same EMS job, two incidents — each was judged before the other existed, so neither could match the other.Probably a prompt-tuning fix in
intelligence.py(scene extraction granularity) rather than a correlator change, but confirm which layer is over-splitting before touching either. Splitting a real multi-scene call is correct behaviour; the bug is splitting one scene into two.Also tracked in
DEFERRED.md.New evidence from a rules-only-period debug_correlation() pull (100 incidents / 2322 orphans, 2026-08-18 16:47Z - 2026-08-20 02:22Z, all on the same talkgroup, all pre the 08-19 00:49Z LLM boundary): this isn't only the single-call-two-scenes case already described above. Separate calls, seconds to a couple minutes apart on the same TGID, describing the same scene, are opening separate incidents because each is judged before the other exists (same root cause, different trigger).
Strongest examples (TG = talkgroup_ids, dt = gap between the two anchor calls):
0213dbcdcall354e5dc3@ 22:34:37 - "I have train 1777 going into 1-2-5 ... track 4 with an EDP and car number 9-2-9-8" vs incident0429a9be("Edp Report at track 4") call7e666a08@ 22:35:14 (dt=37s) - "11-Victor, can you respond to track 4 for a report of EDP on board train 1777 ... car 9298". Same train, same car, same track, 37 seconds apart, two incidents.917f3313("Other", loc="1 Brookville Avenue, 3rd floor") callfcd9c7bd@ 22:10:00 - "What's the address? 1 Brookville Avenue, 3rd floor" vs incident7a79b0d4("Armed Subject") call31754de4@ 22:10:53 (dt=53s, actually overlaps917f3313's own 22:09:18-22:10:00 span) - "The female with the knife is possibly in the kitchen. 10-11 is locked in her room." Same address, same minute, two incidents.cc4f57cf("Theft Report at South Main Street") call7a7ad18e@ 20:44:55 - "any units in the area of Marshalls ... shelf lifter";73ea2660("Suspect Description at in front of Marshalls") callbec391ca@ 20:47:00 (dt=125s from #1) - "red shirt, a gray sweater ... long hair, wearing blue jeans";9357cf81("Shoplifting") callecdecbac@ 20:55:13 - "the merchandise was recovered ... pursue a charge against the individual." Same store, same TG, one continuous shoplifting call chopped into 3 incidents.9e1d8103("Subject Reported", loc=Pleasantville) calld7400576@ 02:52:11 - "Newcastle PD is advising that subject boarded a train. Subject lives in Pleasantville ... departing" vs incident56c38f23("Police", loc=Pleasantville) calld4812418@ 02:53:00 (dt=49s) - "That's all at Newcastle. PD advised. Subject boarded a train. Lives in Pleasantville. Departing train at Pleasantville." Near-verbatim repeat of the same dispatch, split in two.All four are
corr_path=newon both sides (each call became its own incident anchor) rather than the two-scenes-in-one-call pattern in the issue description, so whatever fix lands for this should also cover cross-call near-duplicates on the same TG within a short window, not just intra-call scene splitting.The failure mode appears to have inverted. In the 2026-08-19 dump there is no over-splitting at all — every one of the 6 incidents has exactly one
newcall. What there is instead is heavy over-merging: 4 of 6 incidents are junk chains, one of them 68 calls over 4h09m covering 13 distinct events (now server-26#22).Before doing prompt-tuning work on scene granularity, re-verify that the "10 Parker Street" / "10-Parker, apartment 12" case still reproduces. Tuning extraction to split less while server-26#22 is merging everything would compound the problem.
One mechanism from the review does still point at real same-event splitting and is worth keeping under this issue: the sweep retry budget can expire before the incident it should have matched exists, so a call that arrives slightly early opens its own incident. That is a timing bug rather than a prompt problem, and it would produce exactly the Parker Street symptom.
Full analysis:
CORRELATION_REVIEW_0820.md(project root, not in git).Measured, 2026-08-26 — over-splitting is confirmed and the system has over-corrected
No new AI window this run (morning slot). These numbers come from analysing the two saved correlation dumps that had never been compared:
corr_0819.json(2026-08-20 02:09Z) and10-10pm_8-23.json(2026-08-24 02:06Z).corr_path=newfast/thinThe correlator did not get better between these two windows, it swung to the opposite failure. 08-19 was severe over-merging — one incident absorbed 68 calls. 08-23 is severe over-splitting: two thirds of every "incident" on the board is a single call, and the median incident is one call, which is not an incident at all.
Where the split is being decided
corr_fit_signalisnoneon 50 of 78 calls (64%) — the rule layer finds no signal whatsoever on two thirds of traffic, so almost everything arrives at the tiebreak with nothing to go on.corr_consensus: tiebreak 38, agreed 29, rules_only 9, none 2.corr_llm_action: orphan 31, link 7, none 40.The LLM tier is not passive — it decides 38 of 78 (49%) — and when it decides, it votes against linking better than 4 to 1. That is the proximate mechanism of the over-split. This also updates #35, which recorded the LLM tier as deciding only 3.5% of links; that is stale, it is now roughly half.
Orphans are not sweep-exhausted — they never reached correlation
Of the 23 orphans, 18 are
transcript_too_shortand 1 isgarbage_transcript. Only 3 carry anycorr_sweep_countat all. The orphan pool is gated at the transcript stage, not lost by the sweep. Relevant to #27, which assumes the sweep budget is what strands them.The map, which is the product
45 of 50 incidents have no
location_coords_source. The 5 that do are unverified free-text phrases from radio traffic ("behind Champa Colas", "South High, in front of Scarborough Church"), not resolved addresses. In the 08-19 dump it was 0 of 6. 90%+ of incidents cannot be plotted, which is the thing being sold. Refs #28, #37, #6.Caveat
Different windows, and the 08-19 dump hit its incident-window cap (
incidents_window_exhausted: true) while 08-23 did not, so the incident counts are not strictly like-for-like. The size distribution andcorr_pathshares are, and those are where the swing shows.Static review the same day (#74, #75, #80) finds the sweep path still leaning over-merge, which is consistent: the main path over-splits, the sweep path over-merges. Fixing one direction without measuring the other is how this got here.
Refs #35, #27, #28, #74, #75, #80.
Measured against 3 real dumps (corr_dump_9-7_0437am, corr_dump_9-7_pm, corr_dump_9-12) — the exact mechanism #5 describes does not reproduce
Scripted a scan for the literal pattern: one
call_idappearing incalls_detailof 2+ different incidents. Found 10 hits total (4 / 2 / 4 across the three windows, out of 173/173/181 linked calls each — ~2%).Every single one of the 10 has byte-identical extracted fields (tags, location, units, severity) between its two incident appearances. Example (
corr_dump_9-7_0437am), transcript: "6-David, that's going to be track 4, intox male in his 40s, track 4, Mt. Vernon East. Affirmative. 10-4." — appears in incident3254fd80("Intoxicated Subject at track 4, Mt. Vernon East") ANDfe04dd3a("Police — MTA PD Districts 6/7/11 - Police Dispatch"), both withtags=['intoxicated-subject'] location='track 4, Mt. Vernon East' units=['6-David']— this is the exact DEFERRED.md-cited 9-6 case, still occurring 9-7, now 2 incidents instead of 3.If
intelligence.pyhad actually split this transcript into 2 differently-described scenes, the two incidents would carry different location/tags/units (each scene extracts its own content). They don't — anywhere, in any of the 10 hits. That rules out the mechanism #5's title and body describe (one call → 2+ scenes → 2+ incidents). Zero genuine intra-call multi-scene over-split cases found.EXTRACTION_PROMPT check: current
intelligence.pySCENE DETECTION block still matches DEFERRED.md's description verbatim — default ONE scene, explicit "when unsure, one scene", not-a-new-scene list intact. No drift since the 2026-08-25 tightening.What's actually happening instead: the same single-scene call is being attached to two separate incident records with identical data — a correlator-side duplicate-linking bug, not a scene-extraction bug.
corr_path/corr_consensuson the two appearances are usually identical too (e.g. bothnew/tiebreak, or bothfast/disambig/agreed), consistent with the same call being run through correlation twice and each pass landing on (or creating) a different incident target without removing the call from the other. This produces the same downstream symptom (double-counted call, inflated incident count) DEFERRED.md attributes to #5, but the root cause is different and lives inincident_correlator.py's linking/sweep path, notintelligence.py.Recommendation: close #5 as not reproducing per its stated mechanism — the 2026-08-25 prompt tightening appears to have worked for genuine intra-call scene over-splitting (0/10 candidates confirm it across 3 windows). Open a fresh issue for "same call_id linked into 2 incident records with identical extracted data" (10 instances, 3 windows, evidence above) — that one needs an engineer reading the sweep/re-correlation call_ids append path in
incident_correlator.py, not prompt work. Recommend linking it from the DEFERRED.md #5 row so the row's hypothesis gets corrected rather than left pointing at a mechanism that isn't the live bug.(Investigation only — no code changed.)
Closing per the investigation above: #5's stated mechanism (one call, transcript splits into 2+ scenes, each opens an incident) does not reproduce in 3 live dumps (0/10 candidate hits show differing extracted content between appearances) and the EXTRACTION_PROMPT has not drifted from the 2026-08-25 tightening. The real bug producing the same symptom (a call double-counted across incidents) is a correlator-side duplicate-linking issue, not scene extraction — filed as #131, which supersedes this issue. DEFERRED.md's #5 row updated to point at #131.