Explicit timeouts on every OpenAI and Gemini request (#153)
No AI client set a timeout: OpenAI's SDK waits 600s per attempt and retries twice (timeouts included); google-genai waits forever. A hung provider call had no upper bound, so recorrelation_sweep's 15-minute pipeline hold-back was a guess against an unbounded tail. - config: stt_timeout_seconds (120), openai_timeout_seconds (60), openai_max_retries (1), gemini_timeout_seconds (60). - app/internal/ai_clients.py builds every OpenAI/AsyncOpenAI client with them (transcription, extraction, embeddings, summarizer, vocabulary learner, trip chat). gemini.py's shared client gets HttpOptions(timeout=ms). - A timeout raises like any other provider error, so existing fallbacks are unchanged: rules-only correlation, rules decision on tiebreak, raw transcript on correction, ai_health "transient". - MIN_MINUTES_SINCE_PIPELINE_START is now derived from those timeouts (pipeline_worst_case_seconds, 3-scene budget + 2 min non-AI slack), floor 15: 21 min with the defaults. Loop warns if it reaches the scan window. - tests/test_ai_timeouts.py: client limits, a real hung-socket request that gives up in <5s, timeout fallback per tier, the derived bound, and a guard against any new OpenAI( outside ai_clients.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
d57738a8f6
commit
278edf48dc
@@ -0,0 +1,44 @@
|
||||
"""
|
||||
One place every OpenAI client is built, so none of them runs on SDK-default
|
||||
timeouts (server-26#153).
|
||||
|
||||
The OpenAI SDK's defaults are a 600s read timeout and 2 retries, which it also
|
||||
applies to timeouts: one hung Whisper request could hold a call's pipeline for
|
||||
half an hour, twice over with stt_retry_on_degenerate. That unbounded tail is
|
||||
what recorrelation_sweep.py's MIN_MINUTES_SINCE_PIPELINE_START had to guess
|
||||
against. With these settings it is derived instead (see
|
||||
recorrelation_sweep.pipeline_worst_case_seconds).
|
||||
|
||||
A timeout raises openai.APITimeoutError, an ordinary Exception, so each call
|
||||
site's existing failure path handles it unchanged: ai_health.classify() reads
|
||||
"Request timed out." as transient.
|
||||
|
||||
The Gemini client is built in app/internal/gemini.py with the same intent.
|
||||
"""
|
||||
from typing import Optional
|
||||
|
||||
from app.config import settings
|
||||
|
||||
|
||||
def _timeout(timeout: Optional[float]) -> float:
|
||||
return settings.openai_timeout_seconds if timeout is None else timeout
|
||||
|
||||
|
||||
def openai_client(timeout: Optional[float] = None):
|
||||
"""Synchronous OpenAI client. `timeout` overrides openai_timeout_seconds (STT passes its own)."""
|
||||
from openai import OpenAI # lazy, like every call site before this module
|
||||
return OpenAI(
|
||||
api_key=settings.openai_api_key,
|
||||
timeout=_timeout(timeout),
|
||||
max_retries=settings.openai_max_retries,
|
||||
)
|
||||
|
||||
|
||||
def async_openai_client(timeout: Optional[float] = None):
|
||||
"""Async OpenAI client, same limits as openai_client()."""
|
||||
from openai import AsyncOpenAI
|
||||
return AsyncOpenAI(
|
||||
api_key=settings.openai_api_key,
|
||||
timeout=_timeout(timeout),
|
||||
max_retries=settings.openai_max_retries,
|
||||
)
|
||||
Reference in New Issue
Block a user