Replay: fail fast on a dead AI account; extraction reports to ai_health

First replay (290 calls, 09-22 10:00-12:00 ET) produced 0 incidents and
no errors: every gpt-4o-mini extraction failed and _sync_extract
swallowed it as "no scenes". Same shape as #169 — and the live extraction
tier in /health/ai had no reporter at all, so this has been invisible in
production too.

- intelligence: API failures propagate out of _sync_extract; extract_scenes
  reports them to ai_health ("extraction" tier, billing/dead-model
  classified) and still returns [] so the pipeline degrades as before.
- ai_health: inside a replay sandbox, failures go to the run's own sink
  instead of being dropped.
- replay: aborts after 5 permanent failures on a tier, naming the cause;
  run metrics carry ai_failures; UI shows them.
- replay estimate: audio minutes from started_at/ended_at (no duration
  field exists on call docs).
- ReplayTab exposes the loaded run on window.__drbReplay for in-page
  analysis.

c2-core: 458 pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Logan Cusano
2026-09-26 15:39:01 -04:00
co-authored by Claude Opus 5.5
parent aff3f16d32
commit ec91a9175f
6 changed files with 129 additions and 12 deletions
+11 -1
View File
@@ -324,7 +324,12 @@ function RunDetail({ run }: { run: ReplayRun }) {
useEffect(() => {
setData(null); setError(null);
if (run.status === "running") return;
c2api.getReplayIncidents(run.run_id).then(setData).catch((e) => setError(String(e)));
c2api.getReplayIncidents(run.run_id).then((d) => {
setData(d);
// Exposed for in-page analysis (console / automation) of a run's
// sandbox — the same data this tab renders, nothing more.
(window as unknown as { __drbReplay?: unknown }).__drbReplay = { run, ...d };
}).catch((e) => setError(String(e)));
}, [run.run_id, run.status]);
const m = run.metrics;
@@ -356,6 +361,11 @@ function RunDetail({ run }: { run: ReplayRun }) {
paths: {Object.entries(m.corr_path).map(([k, v]) => `${k} ${v}`).join(" · ")}
</p>
)}
{m?.ai_failures && Object.keys(m.ai_failures).length > 0 && (
<p className="text-xs font-mono text-amber-400">
AI failures: {Object.entries(m.ai_failures).map(([k, v]) => `${k} ×${v}`).join(" · ")}
</p>
)}
{run.errors?.length > 0 && (
<details className="text-xs font-mono text-red-400">
<summary>{run.errors.length} error(s)</summary>
+1
View File
@@ -339,6 +339,7 @@ export interface ReplayMetrics {
corr_consensus: Record<string, number>;
llm_decisions: number;
est_cost_usd: number;
ai_failures?: Record<string, number>;
}
export interface ReplayRun {