Skip to main content
EN · FR Start for free
Menu
Lesson 13 of 22 · Responses API: OpenAI Fundamentals

Rate limits, quotas, and tiers

Lesson 13 of 22 Updated July 29, 2026 Developer
On this page
  1. Rate limits, quotas, and tiers
  2. The tier system
  3. Types of rate limits
  4. Reading rate limit headers
  5. Client-side rate limiting
  6. Strategies for high volumes
  7. Key takeaways

Rate limits, quotas, and tiers

Rate limits define how many requests you can send to the API per minute. Understanding and managing these limits is critical to avoiding service interruptions in production. The nightmare scenario is always the same: an integration that worked flawlessly in development, where you sent three requests per minute, goes live and suddenly processes a queue of several thousand documents. 429 errors arrive in a flood, and half the processing is lost because throughput wasn’t anticipated.

The tier system

OpenAI organizes limits into tiers (levels) based on your billing history. You don’t choose your tier; you unlock it by accumulating spend and waiting out a time window. This progression has an important practical consequence: an account created just for a client demo won’t have the same capacity as your production account, even if the code is identical.

# Tiers determine your limits
# Tier 1: After first successful payment
# Tier 2: After $50 cumulative spend + 7 days
# Tier 3: After $100 cumulative spend + 7 days
# Tier 4: After $250 cumulative spend + 14 days
# Tier 5: After $1000 cumulative spend + 30 days

# Limits increase significantly with each tier

Types of rate limits

Three counters apply simultaneously, and the first one you hit blocks you. Processing short texts will saturate RPM long before TPM; conversely, analyzing long documents will exhaust TPM even if you’ve only sent a few dozen requests. Always check the order of magnitude of your prompts before sizing your parallelism.

# 1. RPM — Requests Per Minute
# 2. TPM — Tokens Per Minute
# 3. RPD — Requests Per Day

# Order of magnitude for Tier 3 (check your actual limits
# at platform.openai.com > Settings > Limits):
# RPM: 5,000 requests/minute
# TPM: 2,000,000 tokens/minute
# RPD: 10,000 requests/day

Reading rate limit headers

You don’t need to guess your consumption: every response carries it in its HTTP headers. The SDK hides them by default to return only the parsed object, but with_raw_response gives you access to the raw response, and parse() reconstructs the usual object. A dashboard fed by these values warns you as you approach the ceiling instead of letting you discover it via an error.

from openai import OpenAI
client = OpenAI()

response = client.responses.create(
    model="gpt-5.6-terra",
    input="Hello!"
)

# Access headers via the raw HTTP response
# Using with_raw_response mode
raw = client.responses.with_raw_response.create(
    model="gpt-5.6-terra",
    input="Hello!"
)

print(f"RPM limit: {raw.headers.get('x-ratelimit-limit-requests')}")
print(f"RPM remaining: {raw.headers.get('x-ratelimit-remaining-requests')}")
print(f"TPM limit: {raw.headers.get('x-ratelimit-limit-tokens')}")
print(f"TPM remaining: {raw.headers.get('x-ratelimit-remaining-tokens')}")
print(f"Reset in: {raw.headers.get('x-ratelimit-reset-requests')}")

response = raw.parse()
print(f"Response: {response.output_text}")

Client-side rate limiting

It’s better to slow yourself down than to get rejected. Two implementations cover most needs.

Limiting throughput with a semaphore

The first combines a semaphore, which caps the number of simultaneous calls, with a delay calculated from the target throughput. In the example, fifty summaries are launched at once but only five proceed in parallel, spaced by the delay corresponding to 100 requests per minute. The calling code remains a simple asyncio.gather: all the discipline is encapsulated in the class.

import asyncio
from openai import AsyncOpenAI

async_client = AsyncOpenAI()

class RateLimiter:
    """Limits the number of simultaneous requests."""

    def __init__(self, max_concurrent: int = 10, rpm: int = 500):
        self.semaphore = asyncio.Semaphore(max_concurrent)
        self.rpm = rpm
        self.delay = 60.0 / rpm  # Delay between each request

    async def call(self, prompt: str) -> str:
        async with self.semaphore:
            await asyncio.sleep(self.delay)
            response = await async_client.responses.create(
                model="gpt-5.6-terra",
                input=prompt
            )
            return response.output_text

async def batch_processing():
    limiter = RateLimiter(max_concurrent=5, rpm=100)

    prompts = [f"Summarize concept {i}" for i in range(50)]
    tasks = [limiter.call(p) for p in prompts]
    results = await asyncio.gather(*tasks)

    for i, r in enumerate(results):
        print(f"Result {i}: {r[:50]}...")

# asyncio.run(batch_processing())

Simple token bucket

The token bucket is more flexible because it allows bursts. The bucket fills at a constant rate and empties with each call: if your application has been idle for a minute, it can restart at full speed for fifty requests, then naturally fall back to the fill rate. This is exactly the behavior you want for a service that alternates between peaks and lulls, and it works in multithreaded environments thanks to the lock.

import time
import threading

class TokenBucket:
    """Rate limiter based on the token bucket algorithm."""

    def __init__(self, tokens_per_second: float, max_capacity: int):
        self.tokens_per_second = tokens_per_second
        self.max_capacity = max_capacity
        self.tokens = max_capacity
        self.last_refill = time.monotonic()
        self.lock = threading.Lock()

    def wait(self, tokens_needed: int = 1):
        """Waits until enough tokens are available."""
        while True:
            with self.lock:
                now = time.monotonic()
                elapsed = now - self.last_refill
                self.tokens = min(
                    self.max_capacity,
                    self.tokens + elapsed * self.tokens_per_second
                )
                self.last_refill = now

                if self.tokens >= tokens_needed:
                    self.tokens -= tokens_needed
                    return

            time.sleep(0.1)

# Usage: max 50 requests per minute
bucket = TokenBucket(tokens_per_second=50/60, max_capacity=50)

def limited_call(prompt: str) -> str:
    bucket.wait(1)
    response = client.responses.create(
        model="gpt-5.6-terra",
        input=prompt
    )
    return response.output_text

Strategies for high volumes

1. Distribute across multiple projects

Each project has its own rate limits. If you hit your ceiling while already on a high tier, distributing calls across multiple projects mechanically multiplies available capacity. The round-robin below illustrates the principle; in production, move the cycle outside the function so it persists between calls.

from openai import OpenAI

# Each project has its own rate limits
clients = [
    OpenAI(project="proj-aaaa"),  # Project 1
    OpenAI(project="proj-bbbb"),  # Project 2
    OpenAI(project="proj-cccc"),  # Project 3
]

def round_robin_call(prompt: str, clients: list) -> str:
    """Distributes calls across multiple projects."""
    import itertools
    client_cycle = itertools.cycle(clients)
    client = next(client_cycle)
    return client.responses.create(
        model="gpt-5.6-terra",
        input=prompt
    ).output_text

2. Use the right model for the right volume

Model choice is also a capacity decision, not just a quality one. Reserving the heaviest models for critical tasks and routing volume to a model with higher limits frees up throughput where you actually need it.

# Economy models (gpt-5.6-luna) have the highest limits
# Use them for high-volume tasks

# The flagship model (gpt-5.6-sol) has the lowest limits
# Reserve it for critical tasks

3. Queuing

Finally, queuing decouples request production from consumption. A single thread dequeues at a fixed pace while the rest of the application submits freely; the caller blocks on an event until its result arrives. This pattern absorbs peaks without ever exceeding the configured throughput.

import queue
import threading

class APIQueue:
    """Queue for API calls."""

    def __init__(self, rpm: int = 100):
        self.queue = queue.Queue()
        self.delay = 60.0 / rpm
        self.running = True
        self.worker = threading.Thread(target=self._process, daemon=True)
        self.worker.start()

    def _process(self):
        while self.running:
            try:
                prompt, future = self.queue.get(timeout=1)
                response = client.responses.create(
                    model="gpt-5.6-terra",
                    input=prompt
                )
                future["result"] = response.output_text
                future["event"].set()
                time.sleep(self.delay)
            except queue.Empty:
                continue

    def submit(self, prompt: str) -> str:
        future = {"result": None, "event": threading.Event()}
        self.queue.put((prompt, future))
        future["event"].wait()
        return future["result"]

Key takeaways

  • Rate limits apply simultaneously in RPM, TPM, and RPD
  • Your tier determines your limits—it increases with your cumulative spend
  • Read x-ratelimit-* headers to know your consumption in real time
  • Implement a client-side rate limiter to avoid 429 errors
  • Distribute load across projects for high volumes
  • Luna offers the highest rate limits for bulk processing
← Previous lessonError Handling and HTTP CodesNext lesson →Retry, Exponential Backoff, and Circuit Breaker