Monitoring costs and usage
On this page
Monitoring costs and usage
API costs can spiral out of control quickly if you don’t watch them. This lesson teaches you how to track, estimate, and control your spending in production. The story repeats itself from project to project: a feature goes live on a Thursday, no one checks the dashboard over the weekend, and Monday morning’s bill matches the entire monthly budget. The steps below help you catch cost drift the moment it happens, not thirty days later.
Understanding billing
Costs are calculated per million tokens, with different prices for input and output. This asymmetry has direct consequences for your design choices: sending a long document as input costs significantly less than having the model generate a long one. Every response returns the exact count for both directions, so you never have to estimate.
from openai import OpenAI
client = OpenAI()
# Every response includes usage metrics
response = client.responses.create(
model="gpt-5.6-terra",
input="Explain OpenAI API billing."
)
usage = response.usage
print(f"Input tokens: {usage.input_tokens}")
print(f"Output tokens: {usage.output_tokens}")
print(f"Total tokens: {usage.total_tokens}")
# Output costs more than input!Cost calculator
Converting these counters into dollars requires a pricing grid. Isolate it in a dedicated structure rather than scattering constants throughout your code: when prices change or you add a model, you only need to modify one line. The rates below are indicative and should be verified during every budget review.
from dataclasses import dataclass
@dataclass
class ModelRate:
input_per_million: float
output_per_million: float
# Approximate prices in USD per million tokens
# Always check current prices on platform.openai.com/pricing
RATES = {
"gpt-5.6-luna": ModelRate(input_per_million=0.20, output_per_million=1.20),
"gpt-5.6-terra": ModelRate(input_per_million=2.0, output_per_million=12.0),
"gpt-5.6-sol": ModelRate(input_per_million=5.0, output_per_million=30.0),
}
def calculate_cost(model: str, input_tokens: int,
output_tokens: int) -> float:
"""Calculates the cost of a call in USD."""
rate = RATES.get(model)
if not rate:
raise ValueError(f"Unknown model: {model}")
input_cost = (input_tokens / 1_000_000) * rate.input_per_million
output_cost = (output_tokens / 1_000_000) * rate.output_per_million
return input_cost + output_cost
# Example
cost = calculate_cost("gpt-5.6-terra", input_tokens=1000, output_tokens=500)
print(f"Cost: ${cost:.6f}")
# Result: Cost: $0.008000Eight-tenths of a cent per call sounds reassuring until you multiply it by the fifty thousand daily calls of a production chatbot—that’s four hundred dollars a day. Always get into the habit of thinking in monthly volumes, never per call.
Real-time cost tracker
Unit calculations are useless unless they are aggregated. This tracker aggregates spending across three axes—by day, by model, and by number of calls—and compares the day’s total to a budget you set. Alerts are graduated: a warning when less than 20% of the budget remains, followed by a hard alert upon exceeding the limit. This intermediate step is valuable because it gives you time to intervene before service is cut off.
import json
from datetime import datetime, date
from collections import defaultdict
class CostTracker:
"""Real-time API cost tracking."""
def __init__(self, daily_budget_usd: float = 10.0):
self.daily_budget = daily_budget_usd
self.costs_per_day: dict[str, float] = defaultdict(float)
self.costs_per_model: dict[str, float] = defaultdict(float)
self.calls_per_day: dict[str, int] = defaultdict(int)
self.total_tokens = 0
def record(self, model: str, input_tokens: int,
output_tokens: int):
"""Records the costs of a call."""
cost = calculate_cost(model, input_tokens, output_tokens)
today = date.today().isoformat()
self.costs_per_day[today] += cost
self.costs_per_model[model] += cost
self.calls_per_day[today] += 1
self.total_tokens += input_tokens + output_tokens
return cost
def remaining_budget(self) -> float:
"""Returns the remaining budget for today."""
today = date.today().isoformat()
return self.daily_budget - self.costs_per_day[today]
def budget_alert(self) -> bool:
"""Checks if the budget is close to the limit."""
remaining = self.remaining_budget()
if remaining <= 0:
print("ALERT: Daily budget exceeded!")
return True
elif remaining < self.daily_budget * 0.2:
print(f"WARNING: ${remaining:.2f} remaining on daily budget")
return True
return False
def report(self) -> dict:
"""Generates a cost report."""
return {
"total_cost_usd": sum(self.costs_per_day.values()),
"cost_today_usd": self.costs_per_day.get(
date.today().isoformat(), 0
),
"remaining_budget_usd": self.remaining_budget(),
"costs_per_model": dict(self.costs_per_model),
"total_tokens": self.total_tokens,
}
# Usage
tracker = CostTracker(daily_budget_usd=5.0)
response = client.responses.create(
model="gpt-5.6-terra",
input="Hello!"
)
cost = tracker.record(
model="gpt-5.6-terra",
input_tokens=response.usage.input_tokens,
output_tokens=response.usage.output_tokens
)
print(f"Cost of this call: ${cost:.6f}")
print(json.dumps(tracker.report(), indent=2))Client with budget control
Monitoring isn’t always enough: faced with a faulty loop calling the API continuously, you need a financial circuit breaker. This client checks the remaining budget before issuing the request and flatly refuses the call if the envelope is exhausted. The service degrades, but in a controlled way without surprise bills. Cost accounting happens afterward, once actual usage is known.
class BudgetControlledClient:
"""Client that refuses calls if the budget is exceeded."""
def __init__(self, daily_budget: float = 10.0):
self.client = OpenAI()
self.tracker = CostTracker(daily_budget_usd=daily_budget)
def create(self, model: str, input: str, **kwargs) -> object:
"""API call with budget control."""
# Check budget before the call
if self.tracker.remaining_budget() <= 0:
raise RuntimeError(
"Daily budget exhausted. "
f"Spent: ${self.tracker.report()['cost_today_usd']:.2f}"
)
response = self.client.responses.create(
model=model,
input=input,
**kwargs
)
cost = self.tracker.record(
model=model,
input_tokens=response.usage.input_tokens,
output_tokens=response.usage.output_tokens
)
# Alert if budget is low
self.tracker.budget_alert()
return response
# Usage
budget_client = BudgetControlledClient(daily_budget=2.0)
try:
response = budget_client.create("gpt-5.6-terra", "Hello!")
print(response.output_text)
except RuntimeError as e:
print(f"Blocked: {e}")Optimizing costs
1. Choosing the right model
The most powerful lever remains routing by complexity. A three-category classification task has no need for the most expensive model: for this task, the bill difference reaches a factor of twenty-five for identical quality. Make complexity an explicit parameter of your function rather than a decision hardcoded into the logic.
# Relative cost per task (approximate, ~300 input tokens / 20 output):
# Simple classification: gpt-5.6-luna ~$0.0001 vs gpt-5.6-sol ~$0.002
# The right model makes a 25x difference in costs!
def optimized_call(prompt: str, complexity: str) -> str:
"""Chooses the most economical suitable model."""
model = {
"simple": "gpt-5.6-luna",
"standard": "gpt-5.6-terra",
"complex": "gpt-5.6-sol",
}.get(complexity, "gpt-5.6-terra")
response = client.responses.create(model=model, input=prompt)
return response.output_text2. Caching responses
The second lever is avoiding paying twice for the same response. An FAQ, product catalog, or standard form generates strictly identical prompts dozens of times a day. The cache key combines the model, input, and parameters—including all of them is essential, otherwise a change in temperature would silently return the old response.
import hashlib
class CachedClient:
"""Client with cache to avoid redundant calls."""
def __init__(self):
self.client = OpenAI()
self.cache: dict[str, str] = {}
def create(self, model: str, input: str, **kwargs) -> str:
# Create a cache key
cache_key = hashlib.md5(
f"{model}:{input}:{json.dumps(kwargs, sort_keys=True)}".encode()
).hexdigest()
if cache_key in self.cache:
print("Cache hit!")
return self.cache[cache_key]
response = self.client.responses.create(
model=model, input=input, **kwargs
)
self.cache[cache_key] = response.output_text
return response.output_text3. Limiting output tokens
Finally, cap the output whenever its length is predictable. Since output is the expensive part, asking for a single word while authorizing two thousand tokens leaves the door open for a verbose response that you will pay for but not use.
# Always set max_output_tokens for predictable tasks
response = client.responses.create(
model="gpt-5.6-terra",
input="Classify this text: positive, negative, or neutral.",
max_output_tokens=10 # Expecting a single word
)Rates as of August 5, 2026 — prices change regularly: before any budget calculation, verify the current grid on the official OpenAI pricing page.
Key takeaways
- Costs depend on the model AND the direction (input vs output)
- Implement a cost tracker with budget alerts
- Use the right model for each task: Luna for volume, Terra by default
- Caching can drastically reduce costs for repetitive requests
- Set
max_output_tokenswhen response length is predictable - Check the OpenAI dashboard for official consumption tracking