Billing and Cost Management

Understand What You’re Paying For

The Responses API has an important billing detail: in a chained conversation, you’re billed for the entire history, even if you don’t explicitly resend it. Understanding this mechanism is essential to controlling your costs in production.

The Cumulative History Trap

With previous_response_id, the server automatically reconstructs the full context. This is convenient for the developer, but it means tokens accumulate:

Growth is quadratic. By the 10th turn of a conversation, input tokens can be 10 to 50 times higher than your initial message.

Reading Costs in the Response

Every response includes a usage object with the exact cost:

{
  "usage": {
    "input_tokens": 450,
    "output_tokens": 280,
    "total_tokens": 730,
    "output_tokens_details": {
      "reasoning_tokens": 200
    },
    "cost_in_nano_usd": 1500000
  }
}

cost_in_nano_usd

The cost_in_nano_usd field gives the cost in nano-dollars:

1,000,000,000 nano-USD = 1 USD

To convert:

cost_usd = response.usage.cost_in_nano_usd / 1_000_000_000
print(f"Cost of this request: ${cost_usd:.6f}")

Reasoning Tokens

The reasoning_tokens in output_tokens_details are billed at the output token rate. They can account for 50 to 80% of output tokens on reasoning models.

Optimization Strategies

Limit Conversation Length

Rather than chaining indefinitely, summarize periodically:

# After N turns, summarize and restart
if turn_count > 5:
    summary = client.responses.create(
        model="grok-4.20-0309-reasoning",
        input="Summarize this conversation in 3 key points.",
        previous_response_id=last_id
    )
    # Restart with the summary as the new context
    new_start = client.responses.create(
        model="grok-4.20-0309-reasoning",
        instructions=f"Previous context: {summary.output[0].content[0].text}",
        input=user_message
    )

Choose the Right Model

Not all models cost the same. For simple tasks, use a less powerful (and cheaper) model. Reserve reasoning models for complex problems.

Control max_output_tokens

Limit the output when you don’t need long responses:

{
  "model": "grok-4.20-0309-reasoning",
  "input": "Answer in one sentence: what is the capital of Japan?",
  "max_output_tokens": 100
}

Use store: false When Possible

Unstored responses aren’t kept on the server. This doesn’t reduce the direct cost but avoids accumulating unnecessary data.

Tracking Costs in Production

Implement systematic tracking:

import json
from datetime import datetime

def log_usage(response, user_id, feature):
    log_entry = {
        "timestamp": datetime.now().isoformat(),
        "user_id": user_id,
        "feature": feature,
        "response_id": response.id,
        "model": response.model,
        "input_tokens": response.usage.input_tokens,
        "output_tokens": response.usage.output_tokens,
        "reasoning_tokens": response.usage.output_tokens_details.reasoning_tokens,
        "cost_nano_usd": response.usage.cost_in_nano_usd,
        "cost_usd": response.usage.cost_in_nano_usd / 1e9
    }
    # Save to your logging system
    save_to_db(log_entry)

Cost Alerts

Set up alert thresholds:

MAX_COST_PER_REQUEST = 0.05  # 5 USD cents

cost = response.usage.cost_in_nano_usd / 1e9
if cost > MAX_COST_PER_REQUEST:
    alert(f"Expensive request detected: ${cost:.4f} (user: {user_id})")

Key Takeaways

• Chained conversations bill for the entire history every turn—quadratic growth
• cost_in_nano_usd / 1 billion = cost in USD
• Reasoning tokens are billed but invisible in the response
• Summarize long conversations to contain costs
• Implement systematic cost tracking per user and per feature