Replay: fail fast on a dead AI account; extraction reports to ai_health

First replay (290 calls, 09-22 10:00-12:00 ET) produced 0 incidents and
no errors: every gpt-4o-mini extraction failed and _sync_extract
swallowed it as "no scenes". Same shape as #169 — and the live extraction
tier in /health/ai had no reporter at all, so this has been invisible in
production too.

- intelligence: API failures propagate out of _sync_extract; extract_scenes
  reports them to ai_health ("extraction" tier, billing/dead-model
  classified) and still returns [] so the pipeline degrades as before.
- ai_health: inside a replay sandbox, failures go to the run's own sink
  instead of being dropped.
- replay: aborts after 5 permanent failures on a tier, naming the cause;
  run metrics carry ai_failures; UI shows them.
- replay estimate: audio minutes from started_at/ended_at (no duration
  field exists on call docs).
- ReplayTab exposes the loaded run on window.__drbReplay for in-page
  analysis.

c2-core: 458 pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Logan Cusano
2026-09-26 15:39:01 -04:00
co-authored by Claude Opus 5.5
parent aff3f16d32
commit ec91a9175f
6 changed files with 129 additions and 12 deletions
+14
View File
@@ -25,6 +25,7 @@ transcription.py) need the exact same judgment call and must not each grow
their own slightly-different copy that drifts.
"""
import asyncio
from contextvars import ContextVar
from datetime import datetime, timezone
from typing import Optional
@@ -59,6 +60,14 @@ def _default_state() -> dict:
_state: dict[str, dict] = {t: _default_state() for t in TIERS}
# Set by a replay run to a list it owns; report_degraded appends there instead
# of touching _state while inside a sandbox (see app/internal/replay.py).
_sandbox_failures: ContextVar[Optional[list]] = ContextVar("drb_ai_sandbox_failures", default=None)
def collect_sandbox_failures(sink: Optional[list]):
return _sandbox_failures.set(sink)
def classify(text: str) -> str:
"""
@@ -112,6 +121,11 @@ async def report_degraded(
if fstore.in_sandbox():
# A replay's rate limits are not a live outage, and must never page
# the AI-alert webhook or flip /health/ai (app/internal/replay.py).
# They are the run's own problem, so they go to the run instead.
sink = _sandbox_failures.get()
if sink is not None:
sink.append({"tier": tier, "provider": provider, "model": model,
"problem": problem, "permanent": permanent})
return
if tier not in _state:
_state[tier] = _default_state()