Two AI spend paths escape config/ai_features entirely, so "AI is off" is not true and #45 cannot measure spend #76

New Issue

2026-08-25T21:30:50-04:00

logan commented

2026-08-25 21:30:50 -04:00

Found by the standing correlation pass on the 2026-08-25 evening unattended run. This one is about money, not correctness.

The operating assumption is that AI is off during the week for cost, and that config/ai_features is the switch. Two paths escape it.

1. Transcript correction is not behind any Firestore flag

transcription.py:208 calls transcript_correction.correct() from inside transcribe_call, gated only by settings.transcript_correction_enabled (config.py:52-53, default True, model gemini-3.6-flash). It also issues a Places Text Search per proposed location via place_verifier.

Consequence: an "STT-only" window is not STT-only. Turning on stt_enabled alone still spends Gemini tokens and Places quota on every call. Killing it needs an env change plus an ansible run — it is not reachable from the admin surface at all.

This is not a bug in isolation (#36 deliberately moved correction into transcription), but it means the cost model of a window is wrong, and #45's seven-day AI spend measurement will attribute this spend to the wrong place.

2. The transcript PATCH route bypasses the flags entirely

_run_extraction_pipeline (upload.py:162-229) checks no flags at all. It is reachable from PATCH /calls/{id}/transcript (calls.py:224), which therefore runs Gemini extraction and correlation with AI globally off.

Bounded in practice — it is a manual admin action, not traffic-driven — but it means "AI is off" is not an enforceable statement about the system, only about the ingest path.

Why this matters now

Gate B5 (#45) requires seven contiguous days of real AI spend to build a cost model, and the board is waiting on the owner for a dollar cap. A spend path that is invisible to the flag that supposedly controls spend makes that measurement unsound before it starts.

Fix

Bring both under get_flags(): gate transcript_correction.correct() on a flag readable from config/ai_features, and have _run_extraction_pipeline resolve flags the same way the upload path does. Add the correction model to whatever #45 ends up measuring.

Refs #45, #50, #36, #37, #57.

Found by the standing correlation pass on the 2026-08-25 evening unattended run. This one is about **money**, not correctness. The operating assumption is that AI is off during the week for cost, and that `config/ai_features` is the switch. Two paths escape it. ## 1. Transcript correction is not behind any Firestore flag `transcription.py:208` calls `transcript_correction.correct()` from inside `transcribe_call`, gated only by `settings.transcript_correction_enabled` (`config.py:52-53`, **default True**, model `gemini-3.6-flash`). It also issues a Places Text Search per proposed location via `place_verifier`. Consequence: an "STT-only" window is not STT-only. Turning on `stt_enabled` alone still spends Gemini tokens and Places quota on every call. Killing it needs an env change plus an ansible run — it is not reachable from the admin surface at all. This is not a bug in isolation (#36 deliberately moved correction into transcription), but it means the cost model of a window is wrong, and #45's seven-day AI spend measurement will attribute this spend to the wrong place. ## 2. The transcript PATCH route bypasses the flags entirely `_run_extraction_pipeline` (`upload.py:162-229`) checks **no flags at all**. It is reachable from `PATCH /calls/{id}/transcript` (`calls.py:224`), which therefore runs Gemini extraction **and** correlation with AI globally off. Bounded in practice — it is a manual admin action, not traffic-driven — but it means "AI is off" is not an enforceable statement about the system, only about the ingest path. ## Why this matters now Gate B5 (#45) requires seven contiguous days of real AI spend to build a cost model, and the board is waiting on the owner for a dollar cap. A spend path that is invisible to the flag that supposedly controls spend makes that measurement unsound before it starts. ## Fix Bring both under `get_flags()`: gate `transcript_correction.correct()` on a flag readable from `config/ai_features`, and have `_run_extraction_pipeline` resolve flags the same way the upload path does. Add the correction model to whatever #45 ends up measuring. Refs #45, #50, #36, #37, #57.

logan referenced this issue

2026-08-26 09:31:22 -04:00

Any signed-in viewer can trigger OpenAI summary spend: incident summary routes use require_service_or_firebase_token, not require_admin_token #81

logan commented

2026-08-26 20:29:24 -04:00

Heads-up for the next unattended run: the fix for this is already written and sitting uncommitted in the working tree - summarizer.py (+7), incidents.py (+6/-2), upload.py (+125/-53, adds _resolve_flags applying the per-system ai_flags override on top of global config/ai_features). It also covers #75 and #81.

It was never committed because the runs that would have finished it died on usage limits (see #82). Do not rewrite it - review, run the c2-core suite, then commit. upload.py is a pipeline file, so tail c2-core logs for ~10 min after the deploy per the standing safeguard in #62.

Heads-up for the next unattended run: the fix for this is **already written and sitting uncommitted** in the working tree - `summarizer.py` (+7), `incidents.py` (+6/-2), `upload.py` (+125/-53, adds `_resolve_flags` applying the per-system `ai_flags` override on top of global `config/ai_features`). It also covers #75 and #81. It was never committed because the runs that would have finished it died on usage limits (see #82). Do not rewrite it - review, run the c2-core suite, then commit. `upload.py` is a pipeline file, so tail c2-core logs for ~10 min after the deploy per the standing safeguard in #62.

logan commented

2026-08-26 20:30:54 -04:00

Heads-up for the next unattended run: the fix for this is already written and sitting uncommitted in the working tree - summarizer.py (+7), incidents.py (+6/-2), upload.py (+125/-53, adds _resolve_flags applying the per-system ai_flags override on top of global config/ai_features). It also covers #75 and #81.

It was never committed because the runs that would have finished it died on usage limits (see #82). Do not rewrite it - review it, run the c2-core suite, then commit. upload.py is a pipeline file, so tail c2-core logs for ~10 min after the deploy per the standing safeguard in #62.

Heads-up for the next unattended run: the fix for this is **already written and sitting uncommitted** in the working tree - `summarizer.py` (+7), `incidents.py` (+6/-2), `upload.py` (+125/-53, adds `_resolve_flags` applying the per-system `ai_flags` override on top of global `config/ai_features`). It also covers #75 and #81. It was never committed because the runs that would have finished it died on usage limits (see #82). Do not rewrite it - review it, run the c2-core suite, then commit. `upload.py` is a pipeline file, so tail c2-core logs for ~10 min after the deploy per the standing safeguard in #62.

logan referenced this issue from a commit

2026-08-27 02:49:19 -04:00

Make "AI is off" true, and stop the transcript PATCH from destroying calls

logan referenced this issue

2026-08-27 02:50:30 -04:00

transcript_correction_enabled is not a pure cost lever: turning it off disables the noise gate and pushes over-merging #83

logan referenced this issue

2026-08-27 02:51:08 -04:00

Any signed-in viewer can trigger OpenAI summary spend: incident summary routes use require_service_or_firebase_token, not require_admin_token #81

logan commented

2026-08-27 02:56:26 -04:00

Fixed and deployed. d18e4f0, live on prod /health at 2026-08-27 06:55 UTC. "AI is off" is now an enforceable statement about the system.

Path 1 — transcript correction. Now behind a new transcript_correction_enabled key in config/ai_features, reachable from the admin surface instead of only via an env var plus an ansible run. The settings.transcript_correction_enabled env check still applies underneath as a hard master off. An STT-only window is now actually STT-only, which is what #45 needs to attribute spend.

Path 2 — the transcript PATCH. _run_extraction_pipeline is gated. More importantly, the route that reaches it now refuses outright.

That second part came out of the pre-push review and is worth recording, because gating extraction alone would have been a worse bug than the one being fixed. PATCH /calls/{id}/transcript wipes tags, severity, location, units, vehicles and embedding, and unlinks the call from every incident, before queueing re-extraction. With correlation off — the standing weekday state — the wipe would have run and the rebuild would not, leaving the call permanently blank and orphaned while the route still answered 200. It now raises 409 before the first write. The wipe and the rebuild are one transaction in intent.

Third path, found in the same function: learn_from_correction was spending with vocabulary_learning_enabled off. Also gated.

Correction: this issue's body describes _run_extraction_pipeline as the reprocess path. It is not — /calls/{id}/reprocess uses _run_intelligence_pipeline, which was already gated. PATCH /calls/{id}/transcript is its only caller, so #7's bulk-reprocess premise was never affected.

Follow-up filed: #83. transcript_correction_enabled is not a pure cost lever — the corrector is also the noise gate that sets not_speech, so turning correction off while correlation is on pushes over-merging. Defaults to True, both flags off in prod, nothing currently exposed. Recorded in DEFERRED.md and in a comment above the flag.

Also in this commit: the summarizer's stale-incident sweep moved outside summaries_enabled. It is pure Firestore with no model call, and gating it meant nothing auto-resolved while AI was off — every incident stayed active forever and the candidate set every correlation reads kept growing.

14 tests added. Suite: 264 passed, 0 failed.

Closing. #45 is now unblocked on this front. #81's auth half remains open.

**Fixed and deployed. `d18e4f0`, live on prod `/health` at 2026-08-27 06:55 UTC.** "AI is off" is now an enforceable statement about the system. **Path 1 — transcript correction.** Now behind a new `transcript_correction_enabled` key in `config/ai_features`, reachable from the admin surface instead of only via an env var plus an ansible run. The `settings.transcript_correction_enabled` env check still applies underneath as a hard master off. An STT-only window is now actually STT-only, which is what #45 needs to attribute spend. **Path 2 — the transcript PATCH.** `_run_extraction_pipeline` is gated. More importantly, the route that reaches it now refuses outright. That second part came out of the pre-push review and is worth recording, because gating extraction alone would have been a worse bug than the one being fixed. `PATCH /calls/{id}/transcript` wipes `tags`, `severity`, `location`, `units`, `vehicles` and `embedding`, and unlinks the call from every incident, **before** queueing re-extraction. With correlation off — the standing weekday state — the wipe would have run and the rebuild would not, leaving the call permanently blank and orphaned while the route still answered `200`. It now raises **409** before the first write. The wipe and the rebuild are one transaction in intent. **Third path, found in the same function:** `learn_from_correction` was spending with `vocabulary_learning_enabled` off. Also gated. **Correction:** this issue's body describes `_run_extraction_pipeline` as the reprocess path. It is not — `/calls/{id}/reprocess` uses `_run_intelligence_pipeline`, which was already gated. `PATCH /calls/{id}/transcript` is its only caller, so #7's bulk-reprocess premise was never affected. **Follow-up filed: #83.** `transcript_correction_enabled` is not a pure cost lever — the corrector is also the noise gate that sets `not_speech`, so turning correction off while correlation is on pushes over-merging. Defaults to True, both flags off in prod, nothing currently exposed. Recorded in `DEFERRED.md` and in a comment above the flag. Also in this commit: the summarizer's stale-incident sweep moved outside `summaries_enabled`. It is pure Firestore with no model call, and gating it meant nothing auto-resolved while AI was off — every incident stayed active forever and the candidate set every correlation reads kept growing. 14 tests added. Suite: 264 passed, 0 failed. Closing. #45 is now unblocked on this front. #81's auth half remains open.

logan closed this issue

2026-08-27 02:56:26 -04:00

logan referenced this issue

2026-08-27 03:05:13 -04:00

Unattended runs died silently on usage limits for 3 cycles - Task Scheduler reported success #82

logan added the area/ai-pipeline area/billing

labels 2026-10-05 02:04:37 -04:00

Sign in to join this conversation.