P0 — standing procedure, runs every evening session.
Correlation quality is the product. The unit of value is the INCIDENT, not the call, so
correlation gets a measured evaluation on a schedule rather than only when something looks wrong.
The full runbook lives at Version 5C/.claude/scheduled/drb-worksession.md (STEP 5) and is
executed by the twice-daily scheduled work session. This issue is the tracker for it.
Procedure
Static pass, every run — drb-correlation-review over incident_correlator.py, llm_correlator.py, intelligence.py, and the upload router scene loop.
Live AI window, evening run only, and only if a correlation-affecting commit is live on /health and has not yet been evaluated against real traffic. Never open a window "just to look".
Arm a server-side failsafe first (sleep 4200 + /opt/drb/ai_off.py) so a crashed agent
cannot leave the window open. If arming fails, do not open the window.
Turn onstt_enabled + correlation_enabled at BOTH levels — global config/ai_features
and the per-system ai_flags on Suffolk a342a757-8c1a-460d-b857-252d2c5e6e94 and Westchester f3bc494e-0951-4a61-b16b-7adc5eb157b1, which survive a global off-switch. Record the prior
values of summaries_enabled / vocabulary_learning_enabled and restore them after.
Wait ~60 min, polling in 9-minute increments (a single long sleep exceeds the tool timeout).
Turn off and verify the read-back shows False, then kill the failsafe.
Dump and review — debug_correlation(limit=100, orphan_hours=2) in-process, saved as Version 5C/correlation-logs-<date>.json so runs are comparable. Trend beats snapshot.
No browser required
PUT /admin/features needs a Firebase admin ID token (require_admin_token in app/internal/auth.py does not accept the service key), which is why earlier automation needed
Chrome. Writing Firestore directly from inside the c2-core container over SSH avoids the HTTP
route entirely and is fully headless.
What each window must answer
Over-splitting rate (#5) — same scene opening two incidents.
Over-merging — one incident absorbing a whole talkgroup.
Orphan count, and whether orphans are still all sweep-"exhausted" (#27).
corr_path / fit_signal distribution, and whether corr_consensus / corr_llm_action populate.
/admin/features accepts only a Firebase admin token. Switching it to require_service_key_or_admin would make automated toggling first-class instead of a
Firestore side-door. Not required for the procedure above — noted so the side-door is a
deliberate choice rather than an accident.
**P0 — standing procedure, runs every evening session.**
Correlation quality is the product. The unit of value is the INCIDENT, not the call, so
correlation gets a measured evaluation on a schedule rather than only when something looks wrong.
The full runbook lives at `Version 5C/.claude/scheduled/drb-worksession.md` (STEP 5) and is
executed by the twice-daily scheduled work session. This issue is the tracker for it.
### Procedure
1. **Static pass, every run** — `drb-correlation-review` over `incident_correlator.py`,
`llm_correlator.py`, `intelligence.py`, and the upload router scene loop.
2. **Live AI window, evening run only, and only if** a correlation-affecting commit is live on
`/health` and has not yet been evaluated against real traffic. Never open a window "just to look".
3. **Arm a server-side failsafe first** (`sleep 4200` + `/opt/drb/ai_off.py`) so a crashed agent
cannot leave the window open. If arming fails, do not open the window.
4. **Turn on** `stt_enabled` + `correlation_enabled` at BOTH levels — global `config/ai_features`
and the per-system `ai_flags` on Suffolk `a342a757-8c1a-460d-b857-252d2c5e6e94` and Westchester
`f3bc494e-0951-4a61-b16b-7adc5eb157b1`, which survive a global off-switch. Record the prior
values of `summaries_enabled` / `vocabulary_learning_enabled` and restore them after.
5. **Wait ~60 min**, polling in 9-minute increments (a single long sleep exceeds the tool timeout).
6. **Turn off and verify the read-back shows False**, then kill the failsafe.
7. **Dump and review** — `debug_correlation(limit=100, orphan_hours=2)` in-process, saved as
`Version 5C/correlation-logs-<date>.json` so runs are comparable. Trend beats snapshot.
### No browser required
`PUT /admin/features` needs a Firebase admin ID token (`require_admin_token` in
`app/internal/auth.py` does not accept the service key), which is why earlier automation needed
Chrome. Writing Firestore directly from inside the `c2-core` container over SSH avoids the HTTP
route entirely and is fully headless.
### What each window must answer
- Over-splitting rate (#5) — same scene opening two incidents.
- Over-merging — one incident absorbing a whole talkgroup.
- Orphan count, and whether orphans are still all sweep-"exhausted" (#27).
- `corr_path` / `fit_signal` distribution, and whether `corr_consensus` / `corr_llm_action` populate.
- Whether map pins land on verified places (#28).
- `tier is DISABLED` in the logs — means a model ID or the billing died again. Highest-priority finding.
Related: #5, #27, #28, #35, #6.
### Follow-up worth doing
`/admin/features` accepts only a Firebase admin token. Switching it to
`require_service_key_or_admin` would make automated toggling first-class instead of a
Firestore side-door. Not required for the procedure above — noted so the side-door is a
deliberate choice rather than an accident.
Window run, 2026-08-25 evening (unattended run). Recording it here since this issue owns the procedure.
Trigger: six correlation-affecting commits landed on main between 2026-08-20 and 2026-08-23 (33a247d…cc038e6, touching incident_correlator.py, intelligence.py, llm_correlator.py). All postdate the CORRELATION_REVIEW_0820.md baseline and none had been evaluated against live traffic.
Procedure as actually executed, which is closer to what this issue should specify than what it currently says:
A server-side failsafe was armed before anything was turned on — a detached 70-minute timer on the server that closes the window regardless of whether the agent survives. Arming it is a precondition; if it fails, the window does not open.
Flags set at both the global config/ai_features level and the per-system ai_flags level, stt_enabled and correlation_enabled only.
summaries_enabled and vocabulary_learning_enabled were already false and were deliberately left untouched, so there is nothing to restore and no learned-state confound in the results.
Read-back confirmed before starting the clock.
Two mechanical notes worth folding into the written procedure:
The scripts live at /opt/drb/aiwin/ on the server and are run via compose run --rm --no-deps with PYTHONPATH=/app and the directory bind-mounted. There is no bind mount of the app source into the running c2-core container in either compose file, so the exec -T c2-core python /app/ai_off.py form assumed by the runbook cannot work as written — the runbook is wrong on that point and should be corrected.
feature_flags caches for 30s but per-system docs are read through doc_get_cached with a 300s TTL. Turning a system on can therefore take up to five minutes to take effect. Turning off is fast, because the global flag is the master switch and beats the per-system value.
Also: tier is DISABLED is emitted once per model per process (llm_correlator.py:296). A container restart re-arms it, so the absence of that line after a restart proves nothing. Any future run of this procedure should record the c2-core container's start time alongside the log grep, or the check is not evidence.
**Window run, 2026-08-25 evening (unattended run).** Recording it here since this issue owns the procedure.
Trigger: six correlation-affecting commits landed on `main` between 2026-08-20 and 2026-08-23 (`33a247d`…`cc038e6`, touching `incident_correlator.py`, `intelligence.py`, `llm_correlator.py`). All postdate the `CORRELATION_REVIEW_0820.md` baseline and none had been evaluated against live traffic.
Procedure as actually executed, which is closer to what this issue should specify than what it currently says:
1. A server-side failsafe was armed **before** anything was turned on — a detached 70-minute timer on the server that closes the window regardless of whether the agent survives. Arming it is a precondition; if it fails, the window does not open.
2. Flags set at both the global `config/ai_features` level and the per-system `ai_flags` level, `stt_enabled` and `correlation_enabled` only.
3. `summaries_enabled` and `vocabulary_learning_enabled` were already `false` and were deliberately left untouched, so there is nothing to restore and no learned-state confound in the results.
4. Read-back confirmed before starting the clock.
Two mechanical notes worth folding into the written procedure:
- The scripts live at `/opt/drb/aiwin/` on the server and are run via `compose run --rm --no-deps` with `PYTHONPATH=/app` and the directory bind-mounted. There is **no** bind mount of the app source into the running `c2-core` container in either compose file, so the `exec -T c2-core python /app/ai_off.py` form assumed by the runbook cannot work as written — the runbook is wrong on that point and should be corrected.
- `feature_flags` caches for 30s but per-system docs are read through `doc_get_cached` with a **300s** TTL. Turning a system *on* can therefore take up to five minutes to take effect. Turning off is fast, because the global flag is the master switch and beats the per-system value.
Also: `tier is DISABLED` is emitted **once per model per process** (`llm_correlator.py:296`). A container restart re-arms it, so the absence of that line after a restart proves nothing. Any future run of this procedure should record the c2-core container's start time alongside the log grep, or the check is not evidence.
First saved correlation dump under this procedure. Unattended run 2026-09-01, against live b722223. Saved as Version 5C/correlation-logs-2026-09-01.json — it is the only one on disk, so this run establishes the baseline and there is no trend to read yet. Every future run compares to this.
Taken with no AI window opened — this is a read of existing state, not spend. AI stayed off throughout.
The numbers
Measure
Value
Incidents
100
Linked calls
187
Orphaned calls
22 (orphan_scan_truncated: false)
Single-call incidents
58 of 100
Median calls/incident
1 · max 7 · max span 25.6 min
Incidents over cap
0 (caps: 40 calls / 120 min)
Incident types
police 70 · other 14 · ems 11 · accident 3 · fire 2
Over-merge: not present. Max 7 calls in one incident, zero incidents at either cap, max span 25.6 minutes against a 120-minute ceiling. Nothing is absorbing a talkgroup. The mechanism-B feedback loop from CORRELATION_REVIEW_0820.md is not firing in this window.
Over-split (#5): not detected. Zero calls appear in more than one incident. Note this probe cannot see the scene-level over-split #5 actually describes, only the call-level consequence — and per #96 the per-scene record is unreliable, so absence of evidence is weak here.
Orphans: 22, and the interesting part is why.orphans_with_transcript: 0 — every single orphan has no transcript. These are not correlation failures at all; correlation was never given anything to work with. That reframes the orphan problem as a transcription-coverage problem and points at #7 (bulk reprocess calls stored without a transcript), not at the correlator. linked_calls_without_transcript is 6.
Pins: 53 of 100 incidents carry a location field, 47 carry none. fallback_titled_incidents: 26 — a quarter of incidents have no real title.
The LLM tier is doing far more than #35 says. 60 of 187 links (32%) have a populated corr_llm_action, and consensus shows 60 tiebreaks. #35 states 3.5%. Do not treat this as a refutation of #35 — per #96 both figures are computed off a per-call blend of scene decisions, so the honest statement is that the instrument disagrees with itself. What is solid: the tier is being consulted, is not disabled, and votes orphan 49 times against link 11 — it is acting predominantly as a brake on linking, not a linker.
No tier is DISABLED condition. Model IDs and billing are alive.
Procedure defect found while running this
The dump could not be taken the way this issue specifies. docker compose run on the VM resolves a stale image, so it executed code older than production and died on a signature change. Filed as #101 — the serious part is that non-signature drift would have produced clean, wrong numbers instead of an error. This dump was taken against the live container instead and is trustworthy; earlier dumps taken via run are of unverified provenance. #57's procedure text needs updating before the next run.
**First saved correlation dump under this procedure. Unattended run 2026-09-01, against live `b722223`.** Saved as `Version 5C/correlation-logs-2026-09-01.json` — it is the only one on disk, so this run establishes the **baseline** and there is no trend to read yet. Every future run compares to this.
Taken with **no AI window opened** — this is a read of existing state, not spend. AI stayed off throughout.
## The numbers
| Measure | Value |
|---|---|
| Incidents | 100 |
| Linked calls | 187 |
| Orphaned calls | 22 (`orphan_scan_truncated: false`) |
| Single-call incidents | 58 of 100 |
| Median calls/incident | 1 · max **7** · max span **25.6 min** |
| Incidents over cap | **0** (caps: 40 calls / 120 min) |
| Incident types | police 70 · other 14 · ems 11 · accident 3 · fire 2 |
`corr_path`: new 101 (54.0%) · fast/thin 56 (29.9%) · fast/disambig 21 (11.2%) · fast/single 4 · unit-continuity 4 · location 1.
`corr_fit_signal`: none 150 · unit_overlap 20 · thin_recency 12 · location_proximity 5.
`corr_consensus`: tiebreak 60 · agreed 55 · rules_only 35 · none 37. `corr_llm_action`: none 127 · **orphan 49 · link 11**.
## Verdicts against the standing failure modes
- **Over-merge: not present.** Max 7 calls in one incident, zero incidents at either cap, max span 25.6 minutes against a 120-minute ceiling. Nothing is absorbing a talkgroup. The mechanism-B feedback loop from `CORRELATION_REVIEW_0820.md` is not firing in this window.
- **Over-split (#5): not detected.** Zero calls appear in more than one incident. Note this probe cannot see the *scene*-level over-split #5 actually describes, only the call-level consequence — and per #96 the per-scene record is unreliable, so absence of evidence is weak here.
- **Orphans: 22, and the interesting part is why.** `orphans_with_transcript: 0` — **every single orphan has no transcript.** These are not correlation failures at all; correlation was never given anything to work with. That reframes the orphan problem as a transcription-coverage problem and points at #7 (bulk reprocess calls stored without a transcript), not at the correlator. `linked_calls_without_transcript` is 6.
- **Pins:** 53 of 100 incidents carry a location field, 47 carry none. `fallback_titled_incidents: 26` — a quarter of incidents have no real title.
- **The LLM tier is doing far more than #35 says.** 60 of 187 links (**32%**) have a populated `corr_llm_action`, and consensus shows 60 tiebreaks. #35 states 3.5%. **Do not treat this as a refutation of #35** — per #96 both figures are computed off a per-call blend of scene decisions, so the honest statement is that the instrument disagrees with itself. What is solid: **the tier is being consulted, is not disabled, and votes `orphan` 49 times against `link` 11** — it is acting predominantly as a brake on linking, not a linker.
- **No `tier is DISABLED` condition.** Model IDs and billing are alive.
## Procedure defect found while running this
The dump could not be taken the way this issue specifies. `docker compose run` on the VM resolves a **stale image**, so it executed code older than production and died on a signature change. Filed as **#101** — the serious part is that non-signature drift would have produced clean, wrong numbers instead of an error. This dump was taken against the live container instead and is trustworthy; earlier dumps taken via `run` are of unverified provenance. **#57's procedure text needs updating before the next run.**
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
P0 — standing procedure, runs every evening session.
Correlation quality is the product. The unit of value is the INCIDENT, not the call, so
correlation gets a measured evaluation on a schedule rather than only when something looks wrong.
The full runbook lives at
Version 5C/.claude/scheduled/drb-worksession.md(STEP 5) and isexecuted by the twice-daily scheduled work session. This issue is the tracker for it.
Procedure
drb-correlation-reviewoverincident_correlator.py,llm_correlator.py,intelligence.py, and the upload router scene loop./healthand has not yet been evaluated against real traffic. Never open a window "just to look".sleep 4200+/opt/drb/ai_off.py) so a crashed agentcannot leave the window open. If arming fails, do not open the window.
stt_enabled+correlation_enabledat BOTH levels — globalconfig/ai_featuresand the per-system
ai_flagson Suffolka342a757-8c1a-460d-b857-252d2c5e6e94and Westchesterf3bc494e-0951-4a61-b16b-7adc5eb157b1, which survive a global off-switch. Record the priorvalues of
summaries_enabled/vocabulary_learning_enabledand restore them after.debug_correlation(limit=100, orphan_hours=2)in-process, saved asVersion 5C/correlation-logs-<date>.jsonso runs are comparable. Trend beats snapshot.No browser required
PUT /admin/featuresneeds a Firebase admin ID token (require_admin_tokeninapp/internal/auth.pydoes not accept the service key), which is why earlier automation neededChrome. Writing Firestore directly from inside the
c2-corecontainer over SSH avoids the HTTProute entirely and is fully headless.
What each window must answer
corr_path/fit_signaldistribution, and whethercorr_consensus/corr_llm_actionpopulate.tier is DISABLEDin the logs — means a model ID or the billing died again. Highest-priority finding.Related: #5, #27, #28, #35, #6.
Follow-up worth doing
/admin/featuresaccepts only a Firebase admin token. Switching it torequire_service_key_or_adminwould make automated toggling first-class instead of aFirestore side-door. Not required for the procedure above — noted so the side-door is a
deliberate choice rather than an accident.
Window run, 2026-08-25 evening (unattended run). Recording it here since this issue owns the procedure.
Trigger: six correlation-affecting commits landed on
mainbetween 2026-08-20 and 2026-08-23 (33a247d…cc038e6, touchingincident_correlator.py,intelligence.py,llm_correlator.py). All postdate theCORRELATION_REVIEW_0820.mdbaseline and none had been evaluated against live traffic.Procedure as actually executed, which is closer to what this issue should specify than what it currently says:
config/ai_featureslevel and the per-systemai_flagslevel,stt_enabledandcorrelation_enabledonly.summaries_enabledandvocabulary_learning_enabledwere alreadyfalseand were deliberately left untouched, so there is nothing to restore and no learned-state confound in the results.Two mechanical notes worth folding into the written procedure:
/opt/drb/aiwin/on the server and are run viacompose run --rm --no-depswithPYTHONPATH=/appand the directory bind-mounted. There is no bind mount of the app source into the runningc2-corecontainer in either compose file, so theexec -T c2-core python /app/ai_off.pyform assumed by the runbook cannot work as written — the runbook is wrong on that point and should be corrected.feature_flagscaches for 30s but per-system docs are read throughdoc_get_cachedwith a 300s TTL. Turning a system on can therefore take up to five minutes to take effect. Turning off is fast, because the global flag is the master switch and beats the per-system value.Also:
tier is DISABLEDis emitted once per model per process (llm_correlator.py:296). A container restart re-arms it, so the absence of that line after a restart proves nothing. Any future run of this procedure should record the c2-core container's start time alongside the log grep, or the check is not evidence.First saved correlation dump under this procedure. Unattended run 2026-09-01, against live
b722223. Saved asVersion 5C/correlation-logs-2026-09-01.json— it is the only one on disk, so this run establishes the baseline and there is no trend to read yet. Every future run compares to this.Taken with no AI window opened — this is a read of existing state, not spend. AI stayed off throughout.
The numbers
orphan_scan_truncated: false)corr_path: new 101 (54.0%) · fast/thin 56 (29.9%) · fast/disambig 21 (11.2%) · fast/single 4 · unit-continuity 4 · location 1.corr_fit_signal: none 150 · unit_overlap 20 · thin_recency 12 · location_proximity 5.corr_consensus: tiebreak 60 · agreed 55 · rules_only 35 · none 37.corr_llm_action: none 127 · orphan 49 · link 11.Verdicts against the standing failure modes
CORRELATION_REVIEW_0820.mdis not firing in this window.orphans_with_transcript: 0— every single orphan has no transcript. These are not correlation failures at all; correlation was never given anything to work with. That reframes the orphan problem as a transcription-coverage problem and points at #7 (bulk reprocess calls stored without a transcript), not at the correlator.linked_calls_without_transcriptis 6.fallback_titled_incidents: 26— a quarter of incidents have no real title.corr_llm_action, and consensus shows 60 tiebreaks. #35 states 3.5%. Do not treat this as a refutation of #35 — per #96 both figures are computed off a per-call blend of scene decisions, so the honest statement is that the instrument disagrees with itself. What is solid: the tier is being consulted, is not disabled, and votesorphan49 times againstlink11 — it is acting predominantly as a brake on linking, not a linker.tier is DISABLEDcondition. Model IDs and billing are alive.Procedure defect found while running this
The dump could not be taken the way this issue specifies.
docker compose runon the VM resolves a stale image, so it executed code older than production and died on a signature change. Filed as #101 — the serious part is that non-signature drift would have produced clean, wrong numbers instead of an error. This dump was taken against the live container instead and is trustworthy; earlier dumps taken viarunare of unverified provenance. #57's procedure text needs updating before the next run.