correlator+summarizer: per-scene call-doc storage, fixes #96 and #114's real fix

Every scene of a multi-scene call correlates independently in upload.py's
scene loop, but every scene's corr_debug was written flat onto the same
shared call doc — scene 2's write silently clobbered scene 1's
corr_path/corr_consensus/etc (#96), and summarizer.py read the whole call's
raw transcript per linked call, mixing text from scenes the incident had
nothing to do with, while ignoring transcript_corrected entirely (#114).

Fix: thread a scene_index from both `for scene in scenes:` loops in
upload.py down through _correlate_with_consensus ->
incident_correlator.preview_correlation/correlate_call -> _build_context ->
ctx["scene_index"]. incident_correlator._apply_and_log now writes, in the
same Firestore call:
  - the existing flat corr_* fields, unchanged (last-scene-wins, the safe
    backward-compatible default for any reader that doesn't know about
    `scenes` yet)
  - a new nested `scenes.<scene_index>` entry with {transcript, incident_id,
    corr_debug}, via doc_set(..., merge=True). Firestore's
    DocumentReference.set(data, merge=True) recursively merges nested map
    fields by key (documented SDK behaviour, not assumed) — a write to
    scenes.1 merges alongside an existing scenes.0 instead of replacing the
    whole `scenes` map.
scene_index defaults to 0 for every caller with no scene concept (the
recorrelation sweep, the no-scenes-extracted orphan-check path), so a plain
single-scene call still gets a one-entry `scenes` map equivalent to reading
its flat fields today.

admin.py's _call_summary exposes the new `scenes` list per call (each entry
carrying the same corr_* field names as the flat fields, so the two shapes
are interchangeable to the tally); the summary tally now iterates each
call's scenes-if-present, else its own flat fields, so a 2-scene call with
two different corr_path values counts as two data points instead of one
blend. New `scene_decision_count` sits next to `linked_call_count` to make
that distinction visible.

summarizer.py's _scene_text_for_incident reads a linked call's `scenes` map
to find the scene(s) whose corr_debug recorded a link into the specific
incident being summarized, joining more than one if several scenes landed
in the same incident. Falls back to transcript_corrected-or-transcript for a
call doc with no `scenes` field (predates this change) — the one-liner half
of #114, worth doing regardless since it stops raw-transcript summaries even
for old-schema docs.

Does not touch #80/#95/#102's existing ctx-threading fixes
(embedding/severity/coords/LLM-prompt-transcript) — correct as-is, out of
scope here.

Tests: 14 new (test_per_scene_call_doc.py, test_summarizer_scene_transcript.py,
additions to test_admin_debug_correlation.py) covering the merge shape,
last-scene-wins flat-field backward compat, the admin tally's per-scene vs
per-call counting (including old-schema fallback), and the summarizer's
scene-specific text selection (including old-schema fallback). Full sandboxed
suite: 364 -> 378 passed, all green.

Fixes server-26#96, server-26#114

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tbknwttzou4s46PAykmtix

drb-c2-core/app/internal/summarizer.py

@@ -15,6 +15,39 @@ from app.internal import firestore as fstore
 from app.config import settings
 
 
+def _scene_sort_key(scene_index: str):
+    """Numeric-first sort so a >=10-scene call's entries still read in order."""
+    return (0, int(scene_index)) if scene_index.isdigit() else (1, scene_index)
+
+
+def _scene_text_for_incident(doc: dict, incident_id: str) -> Optional[str]:
+    """
+    The text of `doc` (a call doc) that actually belongs to `incident_id`.
+
+    server-26#96 records, per scene, which incident_id that scene's
+    correlation decision resolved to (incident_correlator._apply_and_log's
+    `scenes.<index>.incident_id`). Use that to pick only the scene(s) of this
+    call that are genuinely part of this incident, joining more than one if
+    several scenes happened to link into the same incident.
+
+    Falls back to transcript_corrected-or-transcript when the call doc has no
+    `scenes` field (predates server-26#96) or — defensively — when it has one
+    but nothing in it names this incident_id (should not happen for a call_id
+    that's actually in this incident's call_ids, but silently dropping a
+    call's contribution to its own summary would be a worse failure mode than
+    falling back to the whole-call text).
+    """
+    scenes = doc.get("scenes") or {}
+    matched = [
+        scene.get("transcript")
+        for _, scene in sorted(scenes.items(), key=lambda kv: _scene_sort_key(kv[0]))
+        if scene.get("incident_id") == incident_id and scene.get("transcript")
+    ]
+    if matched:
+        return "\n".join(matched)
+    return doc.get("transcript_corrected") or doc.get("transcript")
+
+
 async def summarizer_loop() -> None:
     from app.internal.feature_flags import get_flags
     interval = settings.summary_interval_minutes * 60
@@ -63,12 +96,30 @@ async def _summarize_incident(inc: dict) -> None:
     if not call_ids:
         return
 
-    # Fetch transcripts for all calls in this incident
+    # Fetch transcripts for all calls in this incident.
+    #
+    # server-26#114: a call links into an incident one SCENE at a time (see
+    # incident_correlator._apply_decision / server-26#96's `scenes` map on the
+    # call doc), and the same call_id can appear in more than one incident's
+    # call_ids — once per scene, each scene possibly landing in a different
+    # incident. Reading doc["transcript"] (the whole call, raw) meant an
+    # incident's summary was built partly on text from a DIFFERENT scene of
+    # that call that this incident has nothing to do with, and ignored
+    # transcript_corrected entirely.
+    #
+    # _scene_text_for_incident reads the specific scene(s) whose corr_debug
+    # recorded a link into THIS incident_id. For a call doc that predates
+    # this fix (no `scenes` field) it falls back to
+    # transcript_corrected-or-transcript — the one-liner half of #114, worth
+    # doing even for old-schema docs since it stops raw-transcript summaries.
     transcripts: list[str] = []
     for cid in call_ids:
         doc = await fstore.doc_get("calls", cid)
-        if doc and doc.get("transcript"):
-            transcripts.append(doc["transcript"])
+        if not doc:
+            continue
+        text = _scene_text_for_incident(doc, incident_id)
+        if text:
+            transcripts.append(text)
 
     if not transcripts:
         # No transcripts yet — clear stale flag and wait for next pass