Found in live c2-core logs during the post-deploy watch on the unattended run of 2026-08-27 (d18e4f0). This is happening on production traffic right now, and was already happening before that deploy — it is not a regression from it.
Roughly every inbound call logs:
[WARNING] Duplicate check failed for call <id>: 400 The query requires an index.
...collectionGroups/calls/indexes/... system_id ASC, started_at ASC
Five occurrences in a seven-minute window, against about seven calls. It is not intermittent — dedup is failing on essentially every call.
What this actually costs
dedup.py is what stops the same transmission, heard by two nodes, from being transcribed twice. With its query erroring out, the check degrades to "not a duplicate" and both copies proceed.
While AI is off, the cost is duplicate call documents and inflated counts. The moment an evaluation window opens, every double-heard transmission is paid for twice at Whisper, then extracted twice at Gemini, then correlated twice. That is direct spend, and it also poisons the seven contiguous days of cost data Gate B5 (#45) is waiting on — the figure would be measurably too high for a reason unrelated to the pipeline being measured.
It is also a correctness problem for correlation: two documents for one transmission is exactly the input shape that produces a spurious second incident.
Why it is a WARNING and not an error
dedup.py catches the exception and logs it, so nothing crashes and nothing is loud. That is why this survived undetected — it needed someone reading live logs, not a test. There is no canary on it.
Fix
Add the composite index calls (system_id ASC, started_at ASC) to infra/firestore/firestore.indexes.json and deploy it.
Owner action, and it cannot be done from the authoring machine (per #13 and the DEFERRED.md row): firebase deploy --only firestore:indexes --project discord-radio-bot-461301 from Server/infra/firestore/. Firestore also builds a new composite index asynchronously — it will not take effect the instant the deploy returns.
Related, but not a duplicate
#33 covers the drift between firestore.indexes.json and the live database, and the two redundant ASC duplicates. It does not mention this index and does not describe a live query failing. Worth handling in the same deploy as #33's reconciliation, since both are one firebase deploy. #13 (closed) is the reason a manual deploy step exists at all.
Found in live `c2-core` logs during the post-deploy watch on the unattended run of 2026-08-27 (`d18e4f0`). **This is happening on production traffic right now**, and was already happening before that deploy — it is not a regression from it.
Roughly every inbound call logs:
```
[WARNING] Duplicate check failed for call <id>: 400 The query requires an index.
...collectionGroups/calls/indexes/... system_id ASC, started_at ASC
```
Five occurrences in a seven-minute window, against about seven calls. It is not intermittent — dedup is failing on essentially every call.
## What this actually costs
`dedup.py` is what stops the same transmission, heard by two nodes, from being transcribed twice. With its query erroring out, the check degrades to "not a duplicate" and both copies proceed.
While AI is off, the cost is duplicate call documents and inflated counts. **The moment an evaluation window opens, every double-heard transmission is paid for twice at Whisper, then extracted twice at Gemini, then correlated twice.** That is direct spend, and it also poisons the seven contiguous days of cost data Gate B5 (#45) is waiting on — the figure would be measurably too high for a reason unrelated to the pipeline being measured.
It is also a correctness problem for correlation: two documents for one transmission is exactly the input shape that produces a spurious second incident.
## Why it is a WARNING and not an error
`dedup.py` catches the exception and logs it, so nothing crashes and nothing is loud. That is why this survived undetected — it needed someone reading live logs, not a test. There is no canary on it.
## Fix
Add the composite index `calls (system_id ASC, started_at ASC)` to `infra/firestore/firestore.indexes.json` and deploy it.
**Owner action, and it cannot be done from the authoring machine** (per #13 and the `DEFERRED.md` row): `firebase deploy --only firestore:indexes --project discord-radio-bot-461301` from `Server/infra/firestore/`. Firestore also builds a new composite index asynchronously — it will not take effect the instant the deploy returns.
## Related, but not a duplicate
#33 covers the *drift* between `firestore.indexes.json` and the live database, and the two redundant ASC duplicates. It does not mention this index and does not describe a live query failing. Worth handling in the same deploy as #33's reconciliation, since both are one `firebase deploy`. #13 (closed) is the reason a manual deploy step exists at all.
Refs #33, #13, #45.
The composite index calls (system_id ASC, started_at ASC) was created directly on the c2-server database and has finished building — state READY, all 8 composite indexes now READY. It is also declared in infra/firestore/firestore.indexes.json in commit 187b8c1, so source and the live database agree and the next deploy will not offer to remove it.
This did not need the owner after all. The issue said the fix "cannot be done from the authoring machine" and required a manual firebase deploy. It does not: the production host is a GCP VM already authenticated as the project's compute service account, so gcloud firestore indexes composite create --database=c2-server ... applies it headlessly over SSH. Worth recording, because #51 (no automated deploy path for rules and indexes) has been treating index changes as inherently owner-blocked, and at least the index half of that is not.
Before: 16 Duplicate check failed ... 400 The query requires an index warnings in a 90-minute window, the most recent at 06:30 UTC. After: none since the index went ready. Overnight call volume is thin, so treat the absence as consistent-with-fixed rather than proven on volume — the index state is the real evidence, and the next busy period is the confirmation.
Dedup now actually runs, which means a double-heard transmission is no longer stored twice, no longer pays Whisper and Gemini twice once a window opens, and no longer feeds the correlator the duplicate-document shape that produces a spurious second incident.
#33 is now satisfied in passing — the live index set and the declared set match exactly, 8 for 8. Leaving #33 open is the right call only if it is tracking something beyond this reconciliation; otherwise it can close on this comment.
Refs #33, #13, #45, #51. Commits 187b8c1, deployed at e30d594.
**Fixed and live. Closing.**
The composite index `calls (system_id ASC, started_at ASC)` was created directly on the `c2-server` database and has finished building — state `READY`, all 8 composite indexes now `READY`. It is also declared in `infra/firestore/firestore.indexes.json` in commit `187b8c1`, so source and the live database agree and the next deploy will not offer to remove it.
**This did not need the owner after all.** The issue said the fix "cannot be done from the authoring machine" and required a manual `firebase deploy`. It does not: the production host is a GCP VM already authenticated as the project's compute service account, so `gcloud firestore indexes composite create --database=c2-server ...` applies it headlessly over SSH. Worth recording, because #51 (no automated deploy path for rules and indexes) has been treating index changes as inherently owner-blocked, and at least the index half of that is not.
**Before:** 16 `Duplicate check failed ... 400 The query requires an index` warnings in a 90-minute window, the most recent at 06:30 UTC. **After:** none since the index went ready. Overnight call volume is thin, so treat the absence as consistent-with-fixed rather than proven on volume — the index state is the real evidence, and the next busy period is the confirmation.
Dedup now actually runs, which means a double-heard transmission is no longer stored twice, no longer pays Whisper and Gemini twice once a window opens, and no longer feeds the correlator the duplicate-document shape that produces a spurious second incident.
**#33 is now satisfied in passing** — the live index set and the declared set match exactly, 8 for 8. Leaving #33 open is the right call only if it is tracking something beyond this reconciliation; otherwise it can close on this comment.
Refs #33, #13, #45, #51. Commits `187b8c1`, deployed at `e30d594`.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Found in live
c2-corelogs during the post-deploy watch on the unattended run of 2026-08-27 (d18e4f0). This is happening on production traffic right now, and was already happening before that deploy — it is not a regression from it.Roughly every inbound call logs:
Five occurrences in a seven-minute window, against about seven calls. It is not intermittent — dedup is failing on essentially every call.
What this actually costs
dedup.pyis what stops the same transmission, heard by two nodes, from being transcribed twice. With its query erroring out, the check degrades to "not a duplicate" and both copies proceed.While AI is off, the cost is duplicate call documents and inflated counts. The moment an evaluation window opens, every double-heard transmission is paid for twice at Whisper, then extracted twice at Gemini, then correlated twice. That is direct spend, and it also poisons the seven contiguous days of cost data Gate B5 (#45) is waiting on — the figure would be measurably too high for a reason unrelated to the pipeline being measured.
It is also a correctness problem for correlation: two documents for one transmission is exactly the input shape that produces a spurious second incident.
Why it is a WARNING and not an error
dedup.pycatches the exception and logs it, so nothing crashes and nothing is loud. That is why this survived undetected — it needed someone reading live logs, not a test. There is no canary on it.Fix
Add the composite index
calls (system_id ASC, started_at ASC)toinfra/firestore/firestore.indexes.jsonand deploy it.Owner action, and it cannot be done from the authoring machine (per #13 and the
DEFERRED.mdrow):firebase deploy --only firestore:indexes --project discord-radio-bot-461301fromServer/infra/firestore/. Firestore also builds a new composite index asynchronously — it will not take effect the instant the deploy returns.Related, but not a duplicate
#33 covers the drift between
firestore.indexes.jsonand the live database, and the two redundant ASC duplicates. It does not mention this index and does not describe a live query failing. Worth handling in the same deploy as #33's reconciliation, since both are onefirebase deploy. #13 (closed) is the reason a manual deploy step exists at all.Refs #33, #13, #45.
Fixed and live. Closing.
The composite index
calls (system_id ASC, started_at ASC)was created directly on thec2-serverdatabase and has finished building — stateREADY, all 8 composite indexes nowREADY. It is also declared ininfra/firestore/firestore.indexes.jsonin commit187b8c1, so source and the live database agree and the next deploy will not offer to remove it.This did not need the owner after all. The issue said the fix "cannot be done from the authoring machine" and required a manual
firebase deploy. It does not: the production host is a GCP VM already authenticated as the project's compute service account, sogcloud firestore indexes composite create --database=c2-server ...applies it headlessly over SSH. Worth recording, because #51 (no automated deploy path for rules and indexes) has been treating index changes as inherently owner-blocked, and at least the index half of that is not.Before: 16
Duplicate check failed ... 400 The query requires an indexwarnings in a 90-minute window, the most recent at 06:30 UTC. After: none since the index went ready. Overnight call volume is thin, so treat the absence as consistent-with-fixed rather than proven on volume — the index state is the real evidence, and the next busy period is the confirmation.Dedup now actually runs, which means a double-heard transmission is no longer stored twice, no longer pays Whisper and Gemini twice once a window opens, and no longer feeds the correlator the duplicate-document shape that produces a spurious second incident.
#33 is now satisfied in passing — the live index set and the declared set match exactly, 8 for 8. Leaving #33 open is the right call only if it is tracking something beyond this reconciliation; otherwise it can close on this comment.
Refs #33, #13, #45, #51. Commits
187b8c1, deployed ate30d594.