Automatic API Cache

Cache as a cost-cutting lever

Automatic cache has the biggest impact on your bill, for better or worse. With many providers, you must declare it explicitly in the request, mark the blocks to retain, and manage their lifetime. xAI took a different approach: cache applies automatically to all requests, with no parameters or annotations. You don’t have to activate anything, but you have everything to gain by understanding what triggers it, because a poorly structured prompt never benefits and pays full price on its entire input.

How the cache works

The cache operates on input tokens, and only on them: response generation is always billed at the standard rate. When the start of your prompt exactly reproduces the start of a previously processed prompt, the identical portion is served from the cache at the reduced rate, and only the new part passes at the standard rate.

Three application families benefit immediately. First, a conversation: the system prompt and all previous turns are rigorously identical from one call to the next, only the last question changes, so the cache rate naturally climbs as the exchange lengthens. Next, a fixed-context prompt—a contract, a technical manual, a product repository followed by a variable question—sees its document block cached from the second call onward. Finally, a RAG pipeline keeps constant system instructions; only the retrieved context changes, and if you place it after the instructions, the prefix remains cacheable.

On xAI’s pricing grid, the gap is considerable. On grok-4.3 and grok-4.20-0309, one million cached input tokens costs $0.20 versus $1.25 at the standard rate. On grok-4.5, the same million comes to $0.30 instead of $2.00. In other words, a stable prefix of 5,000 tokens reused one thousand times a day costs you five to seven times less than a prompt reconstructed at every call—for a strictly identical result.

The x-grok-conv-id header

To help the infrastructure associate requests that belong together, the API accepts a dedicated header:

curl -X POST "https://api.x.ai/v1/chat/completions" \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -H "x-grok-conv-id: 550e8400-e29b-41d4-a716-446655440000" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "grok-4.20",
    "messages": [...]
  }'

The x-grok-conv-id header expects a UUID v4. The usage rule is simple: one identifier per conversation or processing flow, reused for all calls that are part of it. This signals that these requests are linked, which increases the probability of cache reuse. The common mistake is generating the UUID at call time rather than at session opening—each turn then receives a different identifier and the benefit disappears. Generate it once, store it with the conversation state:

import uuid

# One UUID per conversation
conv_id = str(uuid.uuid4())

# Use the same UUID for all turns in the conversation
headers = {
    "Authorization": f"Bearer {api_key}",
    "x-grok-conv-id": conv_id,
    "Content-Type": "application/json"
}

Checking cache efficiency

Nothing beats measurement. The cached_tokens field in the usage object, covered in the previous lesson, tells you exactly what was served from the cache:

{
  "usage": {
    "prompt_tokens": 5000,
    "cached_tokens": 4200,
    "completion_tokens": 300
  }
}

Here, 4,200 tokens out of 5,000 come from the cache, a rate of 84%; only 800 tokens were billed at the standard rate. Make it a habit to check this ratio on a sample of real requests after every deployment: if you see a rate near zero on a conversational application, something varies at the head of the prompt, and the next lesson will show you what.

Key takeaways

• Cache is automatic: no configuration necessary
• Use the x-grok-conv-id header with a UUID v4 to group linked requests
• The cached_tokens field in the usage object indicates the number of tokens served from the cache
• Cached input costs 5 to 7 times less: $0.20/M on grok-4.3, $0.30/M on grok-4.5
• The more stable your prompt is at the start of the sequence, the higher the cache rate will be