Explicit timeouts on every OpenAI and Gemini request (#153) #209

2026-10-04T18:00:51-04:00

logan commented

2026-10-04 18:00:51 -04:00

Fixes #153. Related: #169, #196 (AI-tier outages; this does not change billing alerting, only bounds hung requests).

Problem: no OpenAI or Gemini client set a timeout. OpenAI SDK default = 600s/attempt x 3 attempts (it retries timeouts); google-genai default = no timeout. recorrelation_sweep.MIN_MINUTES_SINCE_PIPELINE_START (15) was a guess against that unbounded tail.

Change

New settings: STT_TIMEOUT_SECONDS=120, OPENAI_TIMEOUT_SECONDS=60, OPENAI_MAX_RETRIES=1, GEMINI_TIMEOUT_SECONDS=60.
app/internal/ai_clients.py builds every OpenAI/AsyncOpenAI client (transcription, extraction, embeddings, summarizer, vocabulary learner, trip chat). gemini.py's shared client gets HttpOptions(timeout=ms) (google-genai does not retry by default, so that is the whole bound).
Timeouts raise like any other provider error, so fallbacks are unchanged: decide -> None (rules_only), tiebreak -> rules decision, correction -> raw transcript, extraction -> ai_health extraction transient, embed/summary -> None. classify() reads both timeout messages as transient.
Sweep hold-back is now derived: pipeline_worst_case_seconds() (STT 2 rounds x 2 attempts x 120 + correction 60 + extraction 2x60 + 3 scenes' embeddings 3x2x60 + decide/tiebreak 2x60 + 120s non-AI slack = 1260s), floor 15 -> 21 min with defaults. Sweep loop warns if hold-back >= scan window (60).

Behavior change to note: OpenAI retries drop from 2 to 1 (each retry also re-waits a full timeout on a hang). Override with OPENAI_MAX_RETRIES=2 if 429 bursts show up in ai_health.

Tests: tests/test_ai_timeouts.py (16): client limits, real SDK against a local never-answering socket gives up in <5s, timeout fallback per tier, derived bound, and a guard failing on any new OpenAI( / genai.Client( outside the two factories. test_recorrelation_sweep.py made relative to the derived constant.
c2-core pytest: 541 passed, 0 failed (main baseline: 525 passed). Client untouched.

🤖 Generated with Claude Code

Fixes #153. Related: #169, #196 (AI-tier outages; this does not change billing alerting, only bounds hung requests). **Problem:** no OpenAI or Gemini client set a timeout. OpenAI SDK default = 600s/attempt x 3 attempts (it retries timeouts); google-genai default = no timeout. `recorrelation_sweep.MIN_MINUTES_SINCE_PIPELINE_START` (15) was a guess against that unbounded tail. **Change** - New settings: `STT_TIMEOUT_SECONDS=120`, `OPENAI_TIMEOUT_SECONDS=60`, `OPENAI_MAX_RETRIES=1`, `GEMINI_TIMEOUT_SECONDS=60`. - `app/internal/ai_clients.py` builds every OpenAI/AsyncOpenAI client (transcription, extraction, embeddings, summarizer, vocabulary learner, trip chat). `gemini.py`'s shared client gets `HttpOptions(timeout=ms)` (google-genai does not retry by default, so that is the whole bound). - Timeouts raise like any other provider error, so fallbacks are unchanged: decide -> None (rules_only), tiebreak -> rules decision, correction -> raw transcript, extraction -> ai_health `extraction` transient, embed/summary -> None. `classify()` reads both timeout messages as transient. - Sweep hold-back is now derived: `pipeline_worst_case_seconds()` (STT 2 rounds x 2 attempts x 120 + correction 60 + extraction 2x60 + 3 scenes' embeddings 3x2x60 + decide/tiebreak 2x60 + 120s non-AI slack = 1260s), floor 15 -> **21 min** with defaults. Sweep loop warns if hold-back >= scan window (60). **Behavior change to note:** OpenAI retries drop from 2 to 1 (each retry also re-waits a full timeout on a hang). Override with `OPENAI_MAX_RETRIES=2` if 429 bursts show up in ai_health. **Tests:** `tests/test_ai_timeouts.py` (16): client limits, real SDK against a local never-answering socket gives up in <5s, timeout fallback per tier, derived bound, and a guard failing on any new `OpenAI(` / `genai.Client(` outside the two factories. `test_recorrelation_sweep.py` made relative to the derived constant. c2-core pytest: 541 passed, 0 failed (main baseline: 525 passed). Client untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

logan added 1 commit 2026-10-04 18:00:52 -04:00

Explicit timeouts on every OpenAI and Gemini request (#153 ) 278edf48dc

No AI client set a timeout: OpenAI's SDK waits 600s per attempt and retries
twice (timeouts included); google-genai waits forever. A hung provider call
had no upper bound, so recorrelation_sweep's 15-minute pipeline hold-back was
a guess against an unbounded tail.

- config: stt_timeout_seconds (120), openai_timeout_seconds (60),
  openai_max_retries (1), gemini_timeout_seconds (60).
- app/internal/ai_clients.py builds every OpenAI/AsyncOpenAI client with them
  (transcription, extraction, embeddings, summarizer, vocabulary learner,
  trip chat). gemini.py's shared client gets HttpOptions(timeout=ms).
- A timeout raises like any other provider error, so existing fallbacks are
  unchanged: rules-only correlation, rules decision on tiebreak, raw
  transcript on correction, ai_health "transient".
- MIN_MINUTES_SINCE_PIPELINE_START is now derived from those timeouts
  (pipeline_worst_case_seconds, 3-scene budget + 2 min non-AI slack), floor
  15: 21 min with the defaults. Loop warns if it reaches the scan window.
- tests/test_ai_timeouts.py: client limits, a real hung-socket request that
  gives up in <5s, timeout fallback per tier, the derived bound, and a guard
  against any new OpenAI( outside ai_clients.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

logan referenced this pull request

2026-10-04 18:00:58 -04:00

OpenAI/Gemini clients have no request timeout -- sweep hold-back constants are guessed, not derived #153

logan added 1 commit 2026-10-04 18:29:03 -04:00

Merge branch 'main' into fix/153-ai-request-timeouts 8d9fe01828

logan merged commit a49b814646 into main

2026-10-04 18:29:10 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: logan/server-26#209