No automated deploy path for Firestore rules — they can regress silently after every fix #51

Open
opened 2026-08-23 18:11:14 -04:00 by logan · 4 comments
Owner

Split out of #13. #13 is the one-time operational task (get the current rules live). This issue is the durable gap: even after #13 is done, nothing keeps the deployed ruleset in sync with the repo.

Verified 2026-08-23

  • Server/infra/firestore/firestore.rules, firebase.json and firestore.indexes.json are in source control.
  • .gitea/workflows/deploy.yml has no firebase deploy, firebase-tools, or firestore:rules step. Its only Firestore reference is NEXT_PUBLIC_FIRESTORE_DATABASE as a frontend build-arg (line 65).
  • Grep across every *.md, *.yml, *.yaml, *.sh and Makefile under Server/ returns zero matches for firebase deploy or firestore:rules. No script, no make target, no runbook.
  • The three container images DO auto-deploy on push to main. Firestore rules specifically fall outside that, which makes the gap easy to miss — the reasonable assumption is that everything deploys.

Why this outlives #13

The rules are the only access control on browser direct-reads (useCalls, useIncidents, etc. read Firestore from the client, so auth.py is not in the read path). A security boundary that is edited in the repo but applied by hand will drift again the first time someone changes it and forgets — and the failure is silent in the safe direction for the reviewer (repo looks correct) and unsafe in production.

What done looks like

  • A deploy step that applies firestore.rules (and indexes) from main, ideally in deploy.yml so it shares the existing trigger.
  • Needs a service account with Firebase Rules Admin, added to Gitea secrets.
  • Should FAIL the deploy if the rules fail to apply — a silently-skipped rules deploy is the current state and is worse than a red build.
  • Note #33: firestore.indexes.json has already drifted from the live database and duplicates two indexes. Deploying indexes blindly may error; sequence this after #33 or scope this issue to rules only.

Links

  • #13 — deploy the current rules (operational, do first)
  • #33 — indexes already drifted
  • Board minutes #42 — Gate B blocks charging anyone while the prod tenant boundary is unverified
Split out of #13. **#13 is the one-time operational task** (get the current rules live). This issue is the durable gap: even after #13 is done, nothing keeps the deployed ruleset in sync with the repo. ## Verified 2026-08-23 - `Server/infra/firestore/firestore.rules`, `firebase.json` and `firestore.indexes.json` are in source control. - `.gitea/workflows/deploy.yml` has **no** `firebase deploy`, `firebase-tools`, or `firestore:rules` step. Its only Firestore reference is `NEXT_PUBLIC_FIRESTORE_DATABASE` as a frontend build-arg (line 65). - Grep across every `*.md`, `*.yml`, `*.yaml`, `*.sh` and Makefile under `Server/` returns **zero** matches for `firebase deploy` or `firestore:rules`. No script, no make target, no runbook. - The three container images DO auto-deploy on push to `main`. Firestore rules specifically fall outside that, which makes the gap easy to miss — the reasonable assumption is that everything deploys. ## Why this outlives #13 The rules are the **only** access control on browser direct-reads (`useCalls`, `useIncidents`, etc. read Firestore from the client, so `auth.py` is not in the read path). A security boundary that is edited in the repo but applied by hand will drift again the first time someone changes it and forgets — and the failure is silent in the safe direction for the reviewer (repo looks correct) and unsafe in production. ## What done looks like - A deploy step that applies `firestore.rules` (and indexes) from `main`, ideally in `deploy.yml` so it shares the existing trigger. - Needs a service account with Firebase Rules Admin, added to Gitea secrets. - Should FAIL the deploy if the rules fail to apply — a silently-skipped rules deploy is the current state and is worse than a red build. - Note #33: `firestore.indexes.json` has already drifted from the live database and duplicates two indexes. Deploying indexes blindly may error; sequence this after #33 or scope this issue to rules only. ## Links - #13 — deploy the current rules (operational, do first) - #33 — indexes already drifted - Board minutes #42 — Gate B blocks charging anyone while the prod tenant boundary is unverified
Author
Owner

The concrete casualty: useUnacknowledgedAlerts throws requires an index on every page

drb-frontend/lib/useAlerts.ts useUnacknowledgedAlerts() runs where(org_id ==) where(acknowledged == false) orderBy(triggered_at desc) limit(100) on alert_events. No matching composite index is live, so it fails with FirebaseError: The query requires an index on every page load, and the /watch "Triggered Alerts" tab is dead. This is #13/#51's manual-deploy gap biting a user-visible surface.

File change (branch fix/51-alert-events-index, commit 8a0412b, off main)

Added one entry to infra/firestore/firestore.indexes.json:

{
  "collectionGroup": "alert_events",
  "queryScope": "COLLECTION",
  "fields": [
    { "fieldPath": "acknowledged", "order": "ASCENDING" },
    { "fieldPath": "org_id", "order": "ASCENDING" },
    { "fieldPath": "triggered_at", "order": "DESCENDING" }
  ]
}

Field tuple + triggered_at DESCENDING are copied verbatim from the console create_composite link in the live error, so a gcloud/console create and a firebase deploy converge on the same index (the ASC-vs-DESC split is exactly what caused #33).

Two caveats for whoever reconciles this — do NOT treat as blockers here, they belong to #33:

  1. Scope is COLLECTION, not COLLECTION_GROUP. The hook uses collection(db, "alert_events"), and the error's create_composite payload decodes to queryScope = 1 (COLLECTION). The collectionGroups/ segment in that URL is Firestore's path naming for both scopes, not the scope itself.
  2. The file already carried a near-miss entry commented "the nav badge" (COLLECTION, fields org_id, acknowledged, triggered_at all ASC) that was never deployed. The new entry matches Firestore's own generated spec for the failing query; the old one is now redundant/overlapping. Reconciling the two (and dropping the pre-tenancy live alert_events(acknowledged, triggered_at) index) is #33's job, not this change's.

Not deployed — no GCP creds on the authoring machine, which is the point of this issue.

Deploy options (any one)

1. gcloud (headless over SSH from the prod VM, which is authed as the project SA):

gcloud firestore indexes composite create \
  --project=discord-radio-bot-461301 \
  --database=c2-server \
  --collection-group=alert_events \
  --query-scope=collection \
  --field-config=field-path=acknowledged,order=ascending \
  --field-config=field-path=org_id,order=ascending \
  --field-config=field-path=triggered_at,order=descending

(--query-scope values are lowercase-hyphenated: collection / collection-group / collection-recursive; collection is also the default. Verified against the current gcloud reference.)

2. firebase CLI — from Server/infra/firestore/ (firebase.json pins database: c2-server):

firebase deploy --only firestore:indexes --project discord-radio-bot-461301

This will also prompt to delete the three indexes the file's //drift-2026-08-23 note lists — that note still says answer YES to those three only.

3. Console link from the error:

https://console.firebase.google.com/v1/r/project/discord-radio-bot-461301/firestore/databases/c2-server/indexes?create_composite=Cl1wcm9qZWN0cy9kaXNjb3JkLXJhZGlvLWJvdC00NjEzMDEvZGF0YWJhc2VzL2MyLXNlcnZlci9jb2xsZWN0aW9uR3JvdXBzL2FsZXJ0X2V2ZW50cy9pbmRleGVzL18QARoQCgxhY2tub3dsZWRnZWQQARoKCgZvcmdfaWQQARoQCgx0cmlnZ2VyZWRfYXQQAhoMCghfX25hbWVfXxAC

Index build is a few minutes on an empty/small alert_events; the hook and the Triggered Alerts tab recover on their own once it's READY.

## The concrete casualty: `useUnacknowledgedAlerts` throws `requires an index` on every page `drb-frontend/lib/useAlerts.ts` `useUnacknowledgedAlerts()` runs `where(org_id ==) where(acknowledged == false) orderBy(triggered_at desc) limit(100)` on `alert_events`. No matching composite index is live, so it fails with `FirebaseError: The query requires an index` on every page load, and the `/watch` "Triggered Alerts" tab is dead. This is #13/#51's manual-deploy gap biting a user-visible surface. ### File change (branch `fix/51-alert-events-index`, commit `8a0412b`, off `main`) Added one entry to `infra/firestore/firestore.indexes.json`: ```json { "collectionGroup": "alert_events", "queryScope": "COLLECTION", "fields": [ { "fieldPath": "acknowledged", "order": "ASCENDING" }, { "fieldPath": "org_id", "order": "ASCENDING" }, { "fieldPath": "triggered_at", "order": "DESCENDING" } ] } ``` Field tuple + `triggered_at DESCENDING` are copied verbatim from the console `create_composite` link in the live error, so a `gcloud`/console create and a `firebase deploy` converge on the same index (the ASC-vs-DESC split is exactly what caused #33). **Two caveats for whoever reconciles this — do NOT treat as blockers here, they belong to #33:** 1. **Scope is `COLLECTION`, not `COLLECTION_GROUP`.** The hook uses `collection(db, "alert_events")`, and the error's `create_composite` payload decodes to `queryScope = 1` (COLLECTION). The `collectionGroups/` segment in that URL is Firestore's path naming for both scopes, not the scope itself. 2. The file already carried a near-miss entry commented "the nav badge" (`COLLECTION`, fields `org_id, acknowledged, triggered_at` all ASC) that was never deployed. The new entry matches Firestore's own generated spec for the failing query; the old one is now redundant/overlapping. Reconciling the two (and dropping the pre-tenancy live `alert_events(acknowledged, triggered_at)` index) is #33's job, not this change's. Not deployed — no GCP creds on the authoring machine, which is the point of this issue. ### Deploy options (any one) **1. gcloud (headless over SSH from the prod VM, which is authed as the project SA):** ``` gcloud firestore indexes composite create \ --project=discord-radio-bot-461301 \ --database=c2-server \ --collection-group=alert_events \ --query-scope=collection \ --field-config=field-path=acknowledged,order=ascending \ --field-config=field-path=org_id,order=ascending \ --field-config=field-path=triggered_at,order=descending ``` (`--query-scope` values are lowercase-hyphenated: `collection` / `collection-group` / `collection-recursive`; `collection` is also the default. Verified against the current gcloud reference.) **2. firebase CLI** — from `Server/infra/firestore/` (`firebase.json` pins `database: c2-server`): ``` firebase deploy --only firestore:indexes --project discord-radio-bot-461301 ``` This will also prompt to delete the three indexes the file's `//drift-2026-08-23` note lists — that note still says answer YES to those three only. **3. Console link from the error:** https://console.firebase.google.com/v1/r/project/discord-radio-bot-461301/firestore/databases/c2-server/indexes?create_composite=Cl1wcm9qZWN0cy9kaXNjb3JkLXJhZGlvLWJvdC00NjEzMDEvZGF0YWJhc2VzL2MyLXNlcnZlci9jb2xsZWN0aW9uR3JvdXBzL2FsZXJ0X2V2ZW50cy9pbmRleGVzL18QARoQCgxhY2tub3dsZWRnZWQQARoKCgZvcmdfaWQQARoQCgx0cmlnZ2VyZWRfYXQQAhoMCghfX25hbWVfXxAC Index build is a few minutes on an empty/small `alert_events`; the hook and the Triggered Alerts tab recover on their own once it's `READY`.
Author
Owner

Index still not deployed — this is now the top UI blocker. PR #121 (merged, bc3251e8) declares alert_events (acknowledged ASC, org_id ASC, triggered_at DESC) in infra/firestore/firestore.indexes.json, but there is still no deploy automation (the core of this issue), so the live c2-server DB does not have it. /watch Triggered Alerts + the site-wide useUnacknowledgedAlerts nav badge both stay dead until it is applied.

VM action (GCP service-account authed host): from the repo checkout, Server/infra/firestore/:
firebase deploy --only firestore:indexes --project discord-radio-bot-461301
Answer YES to the 3 deletion prompts per the //drift-2026-08-23 note. This also reconciles the whole index set, which may fix the Archive 503 (#33) if that is a missing calls(org_id, started_at DESC).

**Index still not deployed — this is now the top UI blocker.** PR #121 (merged, bc3251e8) declares `alert_events (acknowledged ASC, org_id ASC, triggered_at DESC)` in `infra/firestore/firestore.indexes.json`, but there is still no deploy automation (the core of this issue), so the live `c2-server` DB does not have it. `/watch` Triggered Alerts + the site-wide `useUnacknowledgedAlerts` nav badge both stay dead until it is applied. **VM action (GCP service-account authed host):** from the repo checkout, `Server/infra/firestore/`: `firebase deploy --only firestore:indexes --project discord-radio-bot-461301` Answer YES to the 3 deletion prompts per the `//drift-2026-08-23` note. This also reconciles the whole index set, which may fix the Archive 503 (#33) if that is a missing `calls(org_id, started_at DESC)`.
logan closed this issue 2026-09-07 23:22:39 -04:00
Author
Owner

Resolved. Root cause was NOT the alert_events index alone — the whole firestore.indexes.json declared ASC-only indexes for orderBy(x,'desc') queries, which Firestore does not reverse-scan for these shapes. Created alert_events (org_id, triggered_at DESC) + (acknowledged, org_id, triggered_at DESC) + calls (org_id, started_at DESC) on c2-server via gcloud (2026-09-08). PR #124 (merged, deploy 7f4d684) rewrites the file to DESC and adds a CI firebase deploy --only firestore:rules,firestore:indexes step so it can't regress silently. /watch Triggered Alerts + the nav alert badge both load now. VM prereq for the CI step: npm i -g firebase-tools once.

Resolved. Root cause was NOT the alert_events index alone — the whole `firestore.indexes.json` declared ASC-only indexes for `orderBy(x,'desc')` queries, which Firestore does not reverse-scan for these shapes. Created `alert_events (org_id, triggered_at DESC)` + `(acknowledged, org_id, triggered_at DESC)` + `calls (org_id, started_at DESC)` on `c2-server` via gcloud (2026-09-08). PR #124 (merged, deploy 7f4d684) rewrites the file to DESC and adds a CI `firebase deploy --only firestore:rules,firestore:indexes` step so it can't regress silently. /watch Triggered Alerts + the nav alert badge both load now. VM prereq for the CI step: `npm i -g firebase-tools` once.
logan reopened this issue 2026-09-13 17:38:41 -04:00
Author
Owner

Reopened per board minutes 2026-09-13 CIO draft (server-26#144): PR #124's "Closes #N" auto-closed this, but the CI firebase deploy step (.gitea/workflows/deploy.yml:131-137) gates on command -v firebase and the VM has no firebase-tools installed -- the step silently no-ops on every deploy. Firestore rules/indexes are NOT actually being applied in prod. See DEFERRED.md row 54. Fix: install firebase-tools on the VM, confirm next deploy log shows a real firebase deploy run, not the WARNING line.

Reopened per board minutes 2026-09-13 CIO draft (server-26#144): PR #124's "Closes #N" auto-closed this, but the CI firebase deploy step (`.gitea/workflows/deploy.yml:131-137`) gates on `command -v firebase` and the VM has no firebase-tools installed -- the step silently no-ops on every deploy. Firestore rules/indexes are NOT actually being applied in prod. See DEFERRED.md row 54. Fix: install firebase-tools on the VM, confirm next deploy log shows a real `firebase deploy` run, not the WARNING line.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: logan/server-26#51