gemini: minimal thinking on correlation, token accounting per call
A day of replay runs (server-26#170) spent ~$5 of Gemini on ~7 two-hour windows (~$0.70 per 290 calls) — several dollars a day per live deployment for correlation alone — and nothing could say where it went (#45). Gemini 3.x thinks by default and bills it as output; the deprecated google-generativeai SDK these calls used cannot set a thinking level. - app/internal/gemini.py: every Gemini call (correlation + transcript correction) goes through google-genai with JSON mode, an explicit thinking level, and logs in/out/thinking tokens. A model that rejects the level is retried without it once and remembered, so the tier is never lost to a config param. API failures still raise for ai_health. - correlator: thinking_level "minimal" (a link/new/orphan choice). transcript correction: "low" until a replay shows minimal is safe. - replay: runs record real Gemini token usage (metrics.gemini_usage), shown in the Replay tab. - requirements: google-genai. c2-core: 474 pass. Frontend typecheck not run (no Node on this box). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
266c958208
commit
0543526eb0
@@ -43,7 +43,6 @@ another equally plausible word.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import re
|
||||
from typing import Any, Optional
|
||||
|
||||
@@ -228,14 +227,11 @@ def build_context_block(context: dict, talkgroup_name: Optional[str]) -> str:
|
||||
|
||||
|
||||
def _sync_gemini(model_name: str, prompt: str) -> dict:
|
||||
import google.generativeai as genai # lazy import — only when needed
|
||||
|
||||
genai.configure(api_key=settings.gemini_api_key)
|
||||
model = genai.GenerativeModel(
|
||||
model_name,
|
||||
generation_config={"response_mime_type": "application/json"},
|
||||
)
|
||||
return json.loads(model.generate_content(prompt).text)
|
||||
from app.internal import gemini
|
||||
# Correction rewrites text against vocabulary; keep a little reasoning
|
||||
# ("low") rather than the correlator's "minimal" until a replay shows
|
||||
# minimal doesn't hurt it.
|
||||
return gemini.generate_json(model_name, prompt, purpose="correction", thinking_level="low")
|
||||
|
||||
|
||||
async def correct(
|
||||
|
||||
Reference in New Issue
Block a user