Updates to reduce firestore calls to try and stay in free tier
### Firestore read reductions
**1. `doc_get_cached()` in `firestore.py` — new 5-min TTL cache**
One place, benefits everything. System and node config documents almost never change during a monitoring session.
**2. System doc: 4 reads → 1 per call**
| Before | After |
|---|---|
| `upload.py` — `doc_get("systems")` for ai_flags | `doc_get_cached` |
| `transcription.py` — `get_vocabulary()` → `doc_get("systems")` | cache hit |
| `intelligence.py` — `get_vocabulary()` → `doc_get("systems")` | cache hit |
| `intelligence.py` — `doc_get("systems")` again for ten_codes | eliminated (reads same cached doc) |
**3. Node doc: cached in `_on_call_start` and `intelligence.py`**
The node is read every call event to get `assigned_system_id` and lat/lon for geocoding. Both now use the cache — node assignments and positions essentially never change at runtime.
**4. Node sweeper: 30s → 90s interval**
The sweeper was doing a full node collection scan 3× more often than necessary — the offline threshold is already 90s. Cuts sweeper reads by 66%.
**5. Vocabulary induction: scans all-time calls → last 7 days**
Previously fetched every ended call for a system (could be thousands). Now scoped to the last 7 days.
> **Note:** The vocabulary induction query `(system_id == X, ended_at >= cutoff)` needs a Firestore
> composite index on `(system_id ASC, ended_at ASC)`. When the induction loop first fires it will log
> an error with a Firebase Console link to create it in one click.
This commit is contained in:
@@ -143,8 +143,8 @@ class MQTTHandler:
|
||||
if not call_id:
|
||||
return
|
||||
|
||||
# Look up assigned system for this node
|
||||
node = await fstore.doc_get("nodes", node_id)
|
||||
# Look up assigned system for this node (cached — assignment rarely changes)
|
||||
node = await fstore.doc_get_cached("nodes", node_id)
|
||||
system_id = node.get("assigned_system_id") if node else None
|
||||
|
||||
started_at_raw = payload.get("started_at")
|
||||
@@ -157,7 +157,7 @@ class MQTTHandler:
|
||||
# Prefer the name from OP25 metadata; fall back to the system config
|
||||
tgid_name = payload.get("tgid_name") or ""
|
||||
if not tgid_name and system_id and payload.get("tgid"):
|
||||
system_doc = await fstore.doc_get("systems", system_id)
|
||||
system_doc = await fstore.doc_get_cached("systems", system_id)
|
||||
if system_doc:
|
||||
tgid_int = int(payload["tgid"])
|
||||
for tg in system_doc.get("config", {}).get("talkgroups", []):
|
||||
|
||||
Reference in New Issue
Block a user