Your OpenAI-compatible endpoint hangs for 180 seconds, then fails. Here is why.

An LLM request that repeatedly stalls for roughly three minutes is a useful debugging clue, not a diagnosis. A client deadline, gateway timeout, upstream delay, buffering, or retries can all contribute. Neither a 178–194 second duration nor a message such as TRANSPORT: Connection error identifies the failing component on its own. Check the actual transport first Streaming HTTP chat completions typically deliver server-sent events (SSE). WebSocket support and automatic transport selection depend on the particular API, client, and version. There is no universal OpenAI-compatible SDK setting named transport: sse. Keys such as api: openai-completions, streamIdleTimeoutMs, and retryPolicy belong to particular applications or wrappers; copying them into an unrelated SDK may do nothing or raise a configuration error. Use your client's documented options and inspect its request logs. If a wrapper really does negotiate WebSockets first, test its documented HTTP/SSE option rather than treating that behavior as common to every SDK. The official Python SDK documents streaming, timeout controls, and retry behavior separately in its README. Check the documentation for the version you installed, including whether it already retries requests, before adding another retry layer. Separate the timeout types A connection timeout limits establishment of a connection. A read timeout limits a wait for incoming data; it is not necessarily a deadline for the whole response. A stream that keeps sending bytes can outlive a read timeout. A correctly implemented read timeout can also fire before the first body chunk, even if headers have arrived. HTTPX distinguishes connect, read, write, and pool timeouts in its timeout documentation. Choose limits for the expected workload: a shorter timeout reduces waiting but may also interrupt a valid long-running generation. A fixed 90-to-180-second ratio is not generally appropriate. Check buffering at each proxy A proxy can delay forwarding upstream chunks. For an nginx HTTP proxy location serving SSE, this is a starting point, not a complete server configuration: # Inside the existing location that proxies to the model backend: proxy_buffering off; proxy_cache off; proxy_read_timeout 90s; # example: maximum gap between upstream reads proxy_read_timeout is measured between successive reads, not across the entire response. nginx can also honor an upstream X-Accel-Buffering: no response header unless configured to ignore it. Other proxies, CDNs, compression layers, or the application itself may still buffer. Changing nginx alone does not guarantee token-by-token delivery. See the nginx proxy module documentation. Retry by error type, with a total deadline String matching an exception for "502" or "timeout" can misclassify failures. Inspect transport exceptions and HTTP status codes. Repeating an unchanged malformed request or invalid credential is generally not useful; some authentication failures require refreshing credentials, not blindly resending them. The following Python 3.11+ example uses HTTPX directly, not SDK-specific configuration. It demonstrates a non-streaming request so that the entire body is consumed inside the deadline. Configure BASE, KEY, and MODEL in your environment; BASE is the API base including /v1 when required. import asyncio import os import random import httpx RETRYABLE_STATUS = {408, 429, 500, 502, 503, 504} async def ask_with_retry(client, *, attempts=3, budget_s=60): if attempts < 1 or budget_s