HTTP 429 and Exponential Backoff

Handling Limit Exceedances

As soon as your application hits its RPM or TPM limits, the Grok API responds with an HTTP 429 status code (Too Many Requests). Read this code for what it is: not an outage, but a flow control signal. The API is saying “slow down,” not “it’s broken.” The difference between a robust integration and a fragile one comes down to how your code interprets this distinction. A mishandled 429 never stays isolated: it cascades through the queue, then into calling services, and ends as an incident visible to the end user.

What Not to Do

The instinctive reaction when a request fails is to resend it immediately. This is exactly the wrong move. You were already over your quota; by resending immediately, you burn an additional call that will also fail, and you extend the blocking period instead of shortening it. An infinite retry makes things worse by tying up threads for nothing. Conversely, treating the 429 as a permanent error and giving up deprives your user of a result that would have arrived a few seconds later. The right approach lies between the two: retry, but wait, and wait longer each time.

Exponential Backoff

The principle is to double the delay after each failed attempt, adding a random variation called jitter.

import time
import random

def call_with_retry(request, max_retries=5):
    for attempt in range(max_retries):
        response = send_request(request)

        if response.status_code != 429:
            return response

        base_delay = 2 ** attempt  # 1, 2, 4, 8, 16 seconds
        jitter = random.uniform(0, base_delay * 0.5)
        delay = base_delay + jitter

        time.sleep(delay)

    raise Exception("Retry limit reached")

Jitter deserves attention because it’s the line most often deleted under the assumption that it complicates the code unnecessarily. Imagine thirty instances of your service receiving a 429 at the same second. Without jitter, they all calculate the same delay and retry at exactly the same moment: you’ve reproduced the load spike identically, one second later, then two, then four. This is the thundering herd phenomenon. Randomization spreads attempts over time and breaks this involuntary synchronization. Note also that the base delay starts at one second: never go below this for the first attempt.

Scaling: Regulate Before Failing

On a high-volume application, reacting to 429s is no longer enough; it’s better to avoid them. A client-side rate limiter caps the number of concurrent calls and enforces a minimum spacing between sends, so outgoing load stays under the ceiling instead of constantly testing it.

import asyncio
from asyncio import Semaphore

# Limit to 25 requests/second (margin under the allowed 30 req/s)
semaphore = Semaphore(25)

async def limited_request(payload):
    async with semaphore:
        response = await send_request_async(payload)
        await asyncio.sleep(1.0 / 25)  # minimum spacing
        return response

Notice the deliberate margin: 25 requests per second instead of the allowed 30. This reserve absorbs network irregularities and traffic bursts without ever hitting the ceiling.

The circuit breaker completes the setup by cutting off calls when failures stack up. While closed, requests pass normally. After a set number of consecutive failures, it opens and locally refuses all requests for a configured duration: you stop hammering an API that is refusing you, and your threads remain available for the rest of the application. When the delay expires, it moves to half-open and lets through a single test request; if it succeeds, normal traffic resumes, otherwise the counter resets. This mechanism protects your service as much as the API on the other end.

Treat 429 as a Metric

One last reflex, often missing from integrations: instrument the 429 rate. This figure does not measure the quality of your retry code; it measures the fit between your tier and your actual traffic. A curve that rises week after week signals undersizing well before users complain, and gives you time to act calmly—distribute load, budget for a tier upgrade—instead of improvising on a Friday night.

Key Takeaways

• HTTP 429 code signals an RPM or TPM limit exceedance
• Exponential backoff with jitter is the strategy recommended by xAI
• In production, complement backoff with a client-side rate limiter and a circuit breaker
• Never retry immediately: wait at least 1 second before the first retry
• Monitor the 429 rate in your metrics as an indicator of undersizing