Scope Gate B3 so the owner can stop being blocked on it (server-26#43)
Board minutes #62 decision 6d bars showing live data to a prospect until #43 is scoped. The conversation count is 0 of 12 with a hard checkpoint on 2026-09-05, so the scoping document is worth more this week than the implementation, which is not due until 2026-09-30. Corrects a premise in #43: the extraction prompt carries no person-name entity field, so "entities are already extracted" does not hold. Redaction has to work on raw free text, and that is most of the estimate. Redaction is specified at write time rather than read time, because the frontend reads Firestore directly and rules cannot mask a field -- redacting only in the API would leave the raw document readable in the browser. EMS exclusion needs a per-talkgroup flag. ai_flags is per-system, and real systems carry EMS alongside police and fire. Refs server-26#43, #42, #62, #66, #85. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
e30d594eea
commit
3df427f914
@@ -0,0 +1,42 @@
|
||||
# Gate B3 (server-26#43) -- Engineering Scope
|
||||
Owner: CTO. Scope only -- no implementation. Ship date unchanged: 2026-09-30.
|
||||
|
||||
## 1. The two defaults, testable
|
||||
- EMS exclusion: for any call whose talkgroup is classified medical, the AI pipeline (Whisper STT + GPT-4o-mini intelligence.py extraction + correlation) must not run. Opt-in only via a per-customer contract flag. Test: upload a call on a talkgroup marked medical, calls/{id}.transcript stays null, no incident_ids.
|
||||
- Name suppression: no surface serving a calls or incidents document (API, frontend render, alert webhook, future export/Discord/API-key tiers) may return an unredacted transcript, summary, or title to any account -- public, comped, or paid -- until an E&O policy is bound (#43 comment 1). Test: same document, two reads -- direct Firestore read and /incidents/{id} API read -- both redacted.
|
||||
|
||||
## 2. Talkgroup-granularity gap
|
||||
feature_flags.py:80-107 (resolve_flags) only layers a per-system ai_flags dict (routers/systems.py:107-129, flat {flag_name: bool}, no talkgroup key) on top of the global default. DEFERREDs own entry for this file says the fix shape is talkgroup_ai_flags: {tgid: {...}} on the system doc, consulted where flag() is built. That field does not exist. Without it, "EMS excluded, rest of the system processed" is not buildable -- the flag is all-on/all-off per system, and most systems mix EMS with police/fire dispatch under one system_id (the exact case BUSINESS_MODEL section 5.5 is trying to protect against). This data-model change is a hard prerequisite, not an enhancement: add talkgroup_ai_flags: {tgid: {stt_enabled, correlation_enabled}} to the system doc, consult it in resolve_flags() before the system-level flag, default every unclassified talkgroup on a system that has at least one confirmed-medical talkgroup to excluded until explicitly classified.
|
||||
|
||||
## 3. Redaction design -- write time, not read time
|
||||
Pick: compute and store a redacted copy alongside the raw one, at extraction/summarization time. Two sentences: the frontend reads Firestore directly for calls/incidents (CLAUDE.md gotcha -- middleware.ts is UX-only, Firestore rules are the real boundary), and Firestore rules can allow/deny a whole document but cannot mask one field inside it -- so a redaction step that only runs inside c2-cores API responses leaves the exact same unredacted transcript/summary/title readable by any authenticated browser via onSnapshot/getDocs against the collection directly. The only enforcement point that actually covers both paths is: the client-readable document never contains the unredacted field. Raw content moves to a field/subcollection excluded from client-facing Firestore rules and readable only server-side by c2-core (satisfies "never deletion, reversible the day a policy binds" -- #43 comment 1).
|
||||
incident.title is template-composed from tag/location/talkgroup (incident_correlator.py:380-389), not LLM freeform -- already name-free by construction, no redaction needed there. The actual carriers are calls.transcript (models.py:165) and the GPT summary (summarizer.py:146-161, built directly from raw transcripts, no name-avoidance instruction today).
|
||||
|
||||
## 4. A premise in #43 does not hold
|
||||
#43s body says "entities are already extracted, so the redaction has a data source to work from." Not true as of this read. intelligence.pys extraction prompt (_PROMPT_TEMPLATE, lines 24-72) has no person-name field -- it extracts tags, incident_type, location, vehicles, units, cleared_units, severity. units is explicitly restricted to "unit IDs or officer numbers... never infer or guess" (line 57) -- radio callsigns, not private-citizen names. There is no structured entity to redact against. Redaction must run against unstructured free text (transcript + GPT summary), via a new regex/NER-style pass with its own unmeasured false-negative rate -- the same class of problem #48 raised about the extractor, one level down, on code that does not exist yet.
|
||||
|
||||
## 5. Surface inventory (complete)
|
||||
- drb-frontend: app/incidents/page.tsx, app/incidents/[id]/page.tsx, app/calls/page.tsx, components/CallRow.tsx, components/CallSpineEntry.tsx -- render title/summary/transcript. Every one is backed by a direct Firestore listener per the section 3 gotcha, not just the page component -- any future onSnapshot/getDocs against calls/incidents inherits the same exposure and must be audited, not assumed covered.
|
||||
- drb-c2-core API: routers/calls.py, routers/incidents.py (JSON responses).
|
||||
- drb-c2-core/app/internal/alerter.py:56,68 -- transcript_snippet (200 chars, raw, unredacted today) written into alert_events and POSTed to the customers own Discord webhook. This is the live, sellable Pro-tier "Alerting" feature (BUSINESS_MODEL section 3.4 item 1) -- highest-priority surface, it is the actual product hook for the beachhead segment.
|
||||
- drb-server-discord-bot: checked app/commands/radio.py, app/commands/trips.py -- embeds today are node status/help/trip content only, no incident transcript/summary rendering exists yet. Nothing to redact today; must inherit this design the day incident-to-Discord posting ships.
|
||||
- drb-telegram-bot: app/handlers/__init__.py is a stub, no incident-surfacing code exists. Same note as above.
|
||||
- Not yet built, but must inherit the design when built: CSV export, Network-tier API access (lib/apiKeys.ts is an in-memory stub per DEFERRED.md).
|
||||
|
||||
## 6. Out of scope for #43
|
||||
- Raw-audio/live-relay exclusion of EMS talkgroups -- the ruling excludes them from the AI pipeline only, not from live audio/Discord voice relay.
|
||||
- Building an accurate NER model -- a heuristic/regex redactor is scope; measuring or improving its accuracy is a follow-on issue (mirrors #48, on the redactor instead of the extractor).
|
||||
- Retroactive redaction of historical calls/incidents already in Firestore (no backfill infra exists -- same unscoped-backfill pattern already logged in DEFERRED.md for _verified_pin). Tracked as a new follow-on issue at ship time, not built now.
|
||||
- A UI for classifying talkgroups as EMS/medical beyond a minimal toggle reusing the existing per-system ai-flags PUT route pattern (routers/systems.py:107).
|
||||
|
||||
## 7. Needs a CEO/owner ruling
|
||||
- Urgent -- is the comped (friends/family) tier suspended today? #43 comment 1 states suppression must hold "on every surface -- public, comped and paid," and #79 comment says no login proceeds until this ships -- but the comped tier is described in BUSINESS_MODEL section 3.2 as already live with "todays live full product," unredacted. Either comped access is in active breach of the ruling right now, or it is meant to be paused pending this ship date. My recommendation: pause comped access to incident detail/transcript views (or accept and log the breach explicitly) until #43 ships -- silently continuing is worse than either choice on record.
|
||||
- How is a talkgroup classified EMS/medical? Recommend: name-pattern heuristic (reusing the existing _TG_SUFFIX_RE EMS/rescue matching in intelligence.py:101-108) as the default classification, manual override in the system editor, and default-exclude on no match rather than default-include -- a false negative here is the exact liability #43 exists to prevent.
|
||||
- Does exclusion/redaction apply retroactively to already-processed calls? Recommend: prospective only for 2026-09-30; backfill is a separate follow-on issue (see section 6).
|
||||
|
||||
## 8. Effort estimate vs 2026-09-30
|
||||
Roughly 6-10 engineering-days, agent-buildable (no human/contractor per GOALS.md), contingent on the section 7 rulings landing quickly -- they gate the design, not just the code:
|
||||
- Talkgroup-flag data model + resolve_flags() wiring: ~1 day.
|
||||
- Minimal EMS-classification toggle (reuse ai-flags PUT pattern): ~1-2 days.
|
||||
- Redacted-copy storage split + Firestore rules change + regex/heuristic redactor + alerter.py snippet redaction + audit of all direct Firestore listeners in frontend: ~4-6 days -- this is the long pole, because section 4 means it is new code, not a wire-up of an existing field.
|
||||
#48 does not block this. #43 comment 1 is explicit: the 200-call accuracy measurement "can no longer decide whether names are published, because they are suppressed regardless. It remains a Gate B condition for other reasons." Sequence independently.
|
||||
Reference in New Issue
Block a user