Timeout and Robustness in Production

Prepare Your Application for the Real World

In production, requests to the Responses API can take time—especially with reasoning models that think extensively before answering. Poor timeout handling can cause silent failures or a degraded user experience.

The 3600-Second Timeout

xAI recommends a minimum timeout of 3600 seconds (1 hour) for requests to reasoning models. This is significantly longer than standard APIs, which impacts your architecture.

Why So Long?

Grok-4.5 reasoning models can use hundreds of thousands of internal reasoning tokens before producing a response. On complex problems (advanced math, code analysis, multi-step reasoning), the model may think for several minutes.

Client-Side Configuration

import httpx
from openai import OpenAI

# OpenAI SDK with extended timeout
client = OpenAI(
    api_key="xai-...",
    base_url="https://api.x.ai/v1",
    timeout=httpx.Timeout(3600.0, connect=10.0)
)

In curl:

curl --max-time 3600 https://api.x.ai/v1/responses \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{...}'

Error Handling

Common Errors

Retry Pattern with Backoff

import time

def call_with_retry(client, max_retries=3, **kwargs):
    for attempt in range(max_retries):
        try:
            return client.responses.create(**kwargs)
        except Exception as e:
            if attempt == max_retries - 1:
                raise
            wait = 2 ** attempt  # 1s, 2s, 4s
            print(f"Error: {e}. Retrying in {wait}s...")
            time.sleep(wait)

Rate Limiting

xAI enforces rate limits based on your plan. Implement a client-side rate limiter:

import time
from collections import deque

class RateLimiter:
    def __init__(self, max_requests_per_minute=60):
        self.max_rpm = max_requests_per_minute
        self.timestamps = deque()

    def wait_if_needed(self):
        now = time.time()
        # Remove timestamps older than one minute
        while self.timestamps and self.timestamps[0] < now - 60:
            self.timestamps.popleft()
        if len(self.timestamps) >= self.max_rpm:
            sleep_time = 60 - (now - self.timestamps[0])
            time.sleep(max(0, sleep_time))
        self.timestamps.append(time.time())

Asynchronous Architecture

For web applications, do not block the main thread for 3600 seconds. Use an asynchronous architecture:

import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI(
    api_key="xai-...",
    base_url="https://api.x.ai/v1"
)

async def process_request(user_input):
    response = await client.responses.create(
        model="grok-4.20-0309-reasoning",
        input=user_input,
        stream=True
    )
    async for event in response:
        if event.type == "response.output_text.delta":
            yield event.delta

Streaming is the best approach for interactive applications: users see the first tokens within seconds, even if full generation takes longer.

Circuit Breaker

To protect your application from xAI API outages:

class CircuitBreaker:
    def __init__(self, failure_threshold=5, reset_timeout=60):
        self.failures = 0
        self.threshold = failure_threshold
        self.reset_timeout = reset_timeout
        self.last_failure = 0
        self.is_open = False

    def call(self, func, *args, **kwargs):
        if self.is_open:
            if time.time() - self.last_failure > self.reset_timeout:
                self.is_open = False
                self.failures = 0
            else:
                raise Exception("Circuit open — API unavailable")

        try:
            result = func(*args, **kwargs)
            self.failures = 0
            return result
        except Exception as e:
            self.failures += 1
            self.last_failure = time.time()
            if self.failures >= self.threshold:
                self.is_open = True
            raise

Key Takeaways

• Configure a minimum timeout of 3600 seconds for reasoning models
• Implement retry with exponential backoff for 429 and 500 errors
• Use streaming for interactive applications — users see tokens immediately
• Handle rate limiting client-side to avoid 429 errors
• Consider a circuit breaker to protect your application from API outages