Admin Replay: re-run the pipeline over past calls in a sandbox (#170)

Correlation has only ever been measured through live AI windows: days of
wall time per change, and the 09-20→22 window was invalidated outright by
unfunded AI accounts (#169). Recordings are kept regardless of AI, so the
traffic to measure against already exists.

- internal/replay.py: runs a time range of real calls through the live
  pipeline code in original order, clock pinned per call, into
  replay_runs/{run_id}/calls|incidents. Modes: audio (re-transcribe),
  transcripts (re-extract), reuse (correlation only from a prior run's
  scenes). Simulates the idle-resolve and orphan-recorrelation sweeps on
  virtual time. No alerts, summaries, vocab, AI-health alerts or pending
  terms. One run at a time, <=5000 calls, <=7 days.
- firestore.py: ContextVar sandbox redirect for calls/incidents.
- clock.py: ContextVar-pinnable now(), used on the correlation path.
- feature_flags.py: ContextVar flag override so replay runs with live AI off.
- upload.py: scene loop extracted to _extract_and_correlate, shared by the
  live pipeline and replay so replay measures the code that runs live.
- resolved_via on every incident resolve, so a real clear can be told
  from the idle timeout — live and in replay.
- routers/replay.py + /admin Replay tab: estimate, start, compare runs,
  drill into incidents with audio.

Reviewed by drb-correlation-review; its leak and fidelity findings are
fixed and covered by tests. c2-core: 456 pass. Frontend typecheck not run
(no Node on the authoring box).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Logan Cusano
2026-09-26 15:23:57 -04:00
co-authored by Claude Opus 5.5
parent e79b8bc37d
commit aff3f16d32
19 changed files with 1976 additions and 101 deletions
+22 -1
View File
@@ -6,7 +6,8 @@ in-memory TTL cache so flag reads don't add a Firestore round-trip to every
call upload.
"""
import time
from typing import Any
from contextvars import ContextVar
from typing import Any, Optional
from app.internal.logger import logger
from app.internal import firestore as fstore
@@ -36,6 +37,21 @@ _DEFAULTS: dict[str, bool] = {
"transcript_correction_enabled": True,
}
# A replay run (app/internal/replay.py) states exactly which AI steps it runs,
# independent of the live switches — the whole point is re-running the pipeline
# while live AI is OFF. ContextVar so the override never reaches a live upload.
_forced: ContextVar[Optional[dict[str, bool]]] = ContextVar("drb_forced_flags", default=None)
def force_flags(flags: Optional[dict[str, bool]]):
"""Override resolve_flags() for the current context. Returns a reset token."""
return _forced.set(flags)
def unforce_flags(token) -> None:
_forced.reset(token)
_cache: dict[str, Any] = {}
_cache_ts: float = 0.0
@@ -211,6 +227,11 @@ async def resolve_flags(system_id: str | None):
"""
from app.internal import firestore as _fstore
forced = _forced.get()
if forced is not None:
full = {k: bool(forced.get(k, False)) for k in _DEFAULTS}
return full, lambda name: full.get(name, False)
flags = await get_flags()
system_ai_flags: dict = {}