OpenAI/Gemini clients have no request timeout -- sweep hold-back constants are guessed, not derived #153

Open
opened 2026-09-14 00:25:33 -04:00 by logan · 0 comments
Owner

Found during drb-correlation-review on server-26#131's fix: neither transcription.py's OpenAI Whisper client (OpenAI(api_key=...), transcription.py:318) nor llm_correlator.py's Gemini call (generate_content(prompt), llm_correlator.py:196) sets a request_options/timeout. Both fall back to SDK defaults -- OpenAI's client default is 600s, doubled by stt_retry_on_degenerate (transcription.py:179) retrying once on a degenerate transcript.

Consequence: recorrelation_sweep.py's MIN_MINUTES_SINCE_PIPELINE_START (server-26#131, currently 15 min) is a guess against this unbounded tail, not a measured bound. A genuinely hung STT or Gemini call can run past any hold-back constant we pick, and the re-correlation sweep would race it again on its next tick -- reopening the exact duplicate-link bug #131 fixed, just for the pipeline's slow-tail population instead of its typical-case population.

Fix shape: set an explicit timeout on both clients (something well inside MAX_RECORDING_SECONDS territory, not SDK defaults), so MIN_MINUTES_SINCE_PIPELINE_START can be derived from a real upper bound instead of picked. Also caps spend on a hung request that would otherwise retry against OpenAI's/Google's default backoff indefinitely.

Not blocking #131 -- filed as the honest follow-up per drb-correlation-review's finding, not fixed in the same pass (matches the project's own precedent of separating instrument fixes from behavior fixes).

Found during `drb-correlation-review` on server-26#131's fix: neither `transcription.py`'s OpenAI Whisper client (`OpenAI(api_key=...)`, `transcription.py:318`) nor `llm_correlator.py`'s Gemini call (`generate_content(prompt)`, `llm_correlator.py:196`) sets a `request_options`/timeout. Both fall back to SDK defaults -- OpenAI's client default is 600s, doubled by `stt_retry_on_degenerate` (`transcription.py:179`) retrying once on a degenerate transcript. Consequence: `recorrelation_sweep.py`'s `MIN_MINUTES_SINCE_PIPELINE_START` (server-26#131, currently 15 min) is a guess against this unbounded tail, not a measured bound. A genuinely hung STT or Gemini call can run past any hold-back constant we pick, and the re-correlation sweep would race it again on its next tick -- reopening the exact duplicate-link bug #131 fixed, just for the pipeline's slow-tail population instead of its typical-case population. Fix shape: set an explicit timeout on both clients (something well inside `MAX_RECORDING_SECONDS` territory, not SDK defaults), so `MIN_MINUTES_SINCE_PIPELINE_START` can be derived from a real upper bound instead of picked. Also caps spend on a hung request that would otherwise retry against OpenAI's/Google's default backoff indefinitely. Not blocking #131 -- filed as the honest follow-up per drb-correlation-review's finding, not fixed in the same pass (matches the project's own precedent of separating instrument fixes from behavior fixes).
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: logan/server-26#153