Cache x-grok-conv-id and Reasoning Timeout

Optimize Costs with Caching

Every request sent to the xAI API is billed at the standard token price. But when you send multiple related requests (same conversation, same topic), part of the context repeats. The x-grok-conv-id header lets xAI identify these related requests and increase the cache hit rate, reducing your costs.

The x-grok-conv-id Header

To enable conversational caching, add a custom header with a unique v4 UUID per conversation:

import uuid

conversation_id = str(uuid.uuid4())

# With the OpenAI SDK
response = await client.chat.completions.create(
    model="grok-4.5",
    messages=[{"role": "user", "content": prompt}],
    extra_headers={
        "x-grok-conv-id": conversation_id
    }
)

All requests sharing the same x-grok-conv-id benefit from an improved cache hit rate. Cached tokens are billed at a reduced rate compared to standard tokens.

Usage Rules

• Generate a new UUID for each conversation or logical session
• Reuse the same UUID for all requests within the same conversation
• Do not reuse a UUID across different conversations (this degrades caching)

# Pattern for a chatbot
class ConversationManager:
    def __init__(self):
        self.conv_id = str(uuid.uuid4())
        self.messages = []

    async def send(self, user_message: str) -> str:
        self.messages.append({"role": "user", "content": user_message})

        response = await client.chat.completions.create(
            model="grok-4.5",
            messages=self.messages,
            extra_headers={"x-grok-conv-id": self.conv_id}
        )

        assistant_message = response.choices[0].message.content
        self.messages.append({"role": "assistant", "content": assistant_message})
        return assistant_message

3600-Second Timeout for Reasoning

Reasoning models (grok-4.20-0309-reasoning and similar) perform internal reasoning before responding. This reasoning can take several minutes for complex problems. The default timeout for most HTTP clients (30-120 seconds) is insufficient.

# Correct configuration for reasoning
from openai import AsyncOpenAI
import httpx

client = AsyncOpenAI(
    api_key=os.getenv("XAI_API_KEY"),
    base_url="https://api.x.ai/v1",
    timeout=httpx.Timeout(
        connect=10.0,     # fast connection
        read=3600.0,      # 1 hour for read
        write=30.0,       # prompt submission
        pool=10.0         # connection wait
    )
)

read=3600.0 is the critical value. This is the maximum delay between when the server starts processing your request and when you receive the response. For a reasoning model that reasons for 5 minutes, you need this margin.

When to Reduce the Timeout

The 3600-second timeout is not always necessary:

• grok-4.5 (non-reasoning): a 120-300 second timeout is usually sufficient
• Simple requests: even with a reasoning model, short questions respond in under a minute
• Test environment: a 60-second timeout allows you to detect issues quickly

# Client for fast requests
fast_client = AsyncOpenAI(
    api_key=os.getenv("XAI_API_KEY"),
    base_url="https://api.x.ai/v1",
    timeout=httpx.Timeout(120.0)
)

# Client for complex reasoning
reasoning_client = AsyncOpenAI(
    api_key=os.getenv("XAI_API_KEY"),
    base_url="https://api.x.ai/v1",
    timeout=httpx.Timeout(3600.0)
)

Using two clients with different timeouts is a common pattern in production.

Note on grok-4.5 and Reasoning

The grok-4.5 model does not return the reasoning_content field in plaintext in the response. Internal reasoning is performed but remains opaque. If you need to see the reasoning process, use use_encrypted_content=True to preserve it between requests without being able to read it, or grok-4.20-0309-reasoning, which exposes more of its process.

Check Cache Impact

To measure cache efficiency, compare the usage field between requests with and without x-grok-conv-id:

response = await client.chat.completions.create(
    model="grok-4.5",
    messages=messages,
    extra_headers={"x-grok-conv-id": conv_id}
)

usage = response.usage
if hasattr(usage, 'prompt_tokens_details'):
    cached = usage.prompt_tokens_details.cached_tokens
    total = usage.prompt_tokens
    print(f"Cache hit: {cached}/{total} tokens ({cached/total*100:.0f}%)")

A high cache hit rate means your context tokens are reused instead of reprocessed, which directly reduces your bill.

Key Takeaways

• x-grok-conv-id with a v4 UUID enables conversational caching for related requests
• Cached tokens are billed at a reduced rate
• A 3600-second timeout is necessary for reasoning models
• Use separate clients for fast requests and reasoning requests
• grok-4.5 does not return reasoning_content in plaintext
• Monitor the cache hit rate in usage.prompt_tokens_details to validate optimization