LLM correlation tier sees the whole-call transcript for every scene, so it cannot separate scenes and pays N times for one prompt #91

New Issue

2026-08-31T02:45:54-04:00

logan commented

2026-08-31 02:45:54 -04:00

Found by the static correlation pass at 29c2fb1, unattended run 2026-08-31.

Not a duplicate: server-26#80 is the same leak on the embedding channel, and server-26#87 was the same leak on the geocode channel (fixed tonight). This is a third channel - the prompt itself.

llm_correlator.py:64 - _call_block builds its block from the whole-call transcript, not from the scene being decided.

One radio call can be split into several scenes, and each scene is correlated separately. But every one of those scenes is sent to the cheap model with the same transcript text, differing only in tags and units. So for an N-scene call the model receives N near-identical prompts, and is being asked to distinguish scenes on evidence that does not vary between them.

Two consequences.

Over-merging. The LLM tier cannot separate the scenes, so it tends to return the same link target for all of them. The tier that exists specifically to catch what the rules get wrong is, on multi-scene calls, structurally unable to.
Wasted spend. N Gemini calls per multi-scene call whose input content is identical. This bills against the same budget server-26#45 is trying to measure, and it is invisible in that measurement because it looks like ordinary per-scene traffic.

Fix: pass the scene's own text span to _call_block rather than the whole-call transcript, so each scene's decision is made on that scene's evidence.

Sequencing. Same root cause as #80 - per-scene evidence is computed and then not used - and the two should be scoped together. Neither should be attempted before #80, because the embedding leak dominates.

Testing note: llm_correlator.py is entirely untested. The existing drb-c2-core suite covers only the gate, dedup and identity paths of incident_correlator.py, so this fix needs its harness built from nothing, and that is most of the cost.

Related: #80, #87, #86, #45. Found alongside the #87 fix; filed separately because it is a different file and a different channel.

**Found by the static correlation pass at `29c2fb1`, unattended run 2026-08-31.** Not a duplicate: server-26#80 is the same leak on the *embedding* channel, and server-26#87 was the same leak on the *geocode* channel (fixed tonight). This is a third channel - the prompt itself. `llm_correlator.py:64` - `_call_block` builds its block from the **whole-call transcript**, not from the scene being decided. One radio call can be split into several scenes, and each scene is correlated separately. But every one of those scenes is sent to the cheap model with the **same transcript text**, differing only in tags and units. So for an N-scene call the model receives N near-identical prompts, and is being asked to distinguish scenes on evidence that does not vary between them. **Two consequences.** 1. **Over-merging.** The LLM tier cannot separate the scenes, so it tends to return the same `link` target for all of them. The tier that exists specifically to catch what the rules get wrong is, on multi-scene calls, structurally unable to. 2. **Wasted spend.** N Gemini calls per multi-scene call whose input content is identical. This bills against the same budget server-26#45 is trying to measure, and it is invisible in that measurement because it looks like ordinary per-scene traffic. **Fix:** pass the scene's own text span to `_call_block` rather than the whole-call transcript, so each scene's decision is made on that scene's evidence. **Sequencing.** Same root cause as #80 - per-scene evidence is computed and then not used - and the two should be scoped together. Neither should be attempted before #80, because the embedding leak dominates. **Testing note:** `llm_correlator.py` is **entirely untested**. The existing `drb-c2-core` suite covers only the gate, dedup and identity paths of `incident_correlator.py`, so this fix needs its harness built from nothing, and that is most of the cost. Related: #80, #87, #86, #45. Found alongside the #87 fix; filed separately because it is a different file and a different channel.

logan referenced this issue

2026-09-01 02:43:34 -04:00

Non-primary scenes inherit the primary scene's severity from the call doc, so bdb57ae closed one of three leaks and the other two now carry the weight alone #95

logan added the area/correlation

labels 2026-10-05 02:04:42 -04:00

logan referenced this issue