Migration from Legacy and Best Practices

From chat/completions to responses

Many production Grok integrations still rely on /v1/chat/completions, often without an explicit decision: the OpenAI SDK was already in place, you just had to change the base URL and key for everything to work. This lesson walks you through the switch to /v1/responses, the primary endpoint, then broadens the scope to practices that keep an integration healthy over the long haul, migration or not.

Why leave the legacy endpoint

/v1/chat/completions remains maintained and your requests will continue to work. The problem isn’t outage, it’s stagnation: future development focuses exclusively on /v1/responses, fixes specific to the legacy endpoint are treated with lower priority, and certain advanced capabilities—agents, native tools—are simply not available there.

So there’s no urgency if your application runs correctly today. There is, however, an implicit deadline: every feature you’ll want to adopt in the coming months will require having migrated. Better to handle this work calmly than when a product need forces it.

What changes between the two endpoints

The legacy endpoint expects the classic messages format you know:

# Legacy: /v1/chat/completions
payload = {
    "model": "grok-4.20-0309",
    "messages": [
        {"role": "system", "content": "Instructions..."},
        {"role": "user", "content": "Question"}
    ]
}

/v1/responses adopts an enriched format that natively supports tools, agents, and complex conversational flows; the xAI documentation provides the details. Practically, the main consequence is on the client side: if you use the OpenAI SDK to talk to the Grok API, migration may involve changing SDKs or switching to the native xAI SDK. Check this point before estimating workload, because it often shifts the project from “replace a URL” to “replace a dependency”.

Running the migration

Start with an inventory, and take it seriously. Direct HTTP calls are easy to find, but the API is rarely invoked from a single place: SDKs—OpenAI, native xAI—often coexist in the same codebase, intermediate services like proxies or gateways may rewrite requests, and batch scripts or data pipelines live their lives far from application code. These are the ones you forget, and they’re the ones that break three weeks after migration.

Then migrate in small chunks, starting with the least critical integrations. A feature flag lets you switch one code path at a time and, crucially, roll back without redeployment:

# Feature flag for progressive migration
if feature_flags.get("use_responses_api"):
    response = call_responses_api(prompt)
else:
    response = call_chat_completions(prompt)

This flag only makes sense if the rest of your code ignores which endpoint responded. Hence the third piece of the setup: a normalization function that brings both response formats back to a single structure, so the migration stays confined to a single layer.

def normalize_response(raw_response, endpoint):
    """Normalizes responses regardless of the endpoint used."""
    if endpoint == "responses":
        return {
            "content": raw_response["output"]["content"],
            "usage": raw_response["usage"]
        }
    else:  # chat/completions
        return {
            "content": raw_response["choices"][0]["message"]["content"],
            "usage": raw_response["usage"]
        }

Practices that survive migrations

This normalization function illustrates a broader principle: isolate your code behind an abstract interface, even if you use only one provider. What costs dearly during an endpoint, version, or provider change is never the call itself, it’s the number of places that depend on it.

class ClientLLM:
    def completer(self, messages, model=None):
        """Abstract interface for LLM calls."""
        raise NotImplementedError

class ClientGrok(ClientLLM):
    def completer(self, messages, model=None):
        model = model or GROK_MODEL
        # Grok-specific implementation
        pass

Then log every call in a structured way. Five metadata points suffice, and they only hold value together: the model used and its version, the number of tokens input, output, and served from cache, the request cost, latency—TTFT and total duration—and finally the HTTP status code. Without these traces, a quality incident after migration becomes a debate of impressions. With them, you compare two populations of requests and decide in ten minutes.

Finally, build a regression test suite for your critical prompts, to run before any model or endpoint switch. It checks three distinct things: that the response respects the expected format, JSON or Markdown as the case may be; that key information actually appears there; and that your guardrails still hold. A more performant model that subtly bypasses your safety instructions is a regression, even if its responses seem better.

In steady state, four signals deserve permanent monitoring: the error rate per endpoint and per model, quality drift measured by automatic or human evaluation, deprecation notices published by xAI, and your consumption relative to your current tier limits. The last two are the only ones that warn you before the incident rather than after.

Key takeaways

• The /v1/chat/completions endpoint is legacy and no longer receives new features
• Migrate progressively to /v1/responses using feature flags
• Create an abstraction layer to normalize responses between the two endpoints
• Use pinned versions in production and aliases in development
• Maintain prompt regression tests and continuous monitoring
• Centralize model configuration to facilitate future migrations

Test your knowledge

Tiers, tokens, cache, lifecycle: operating Grok in production.

1. How do account rate limits evolve?

Answer: Via automatic tiers: consumption drives up RPM/TPM per model—and Enterprise negotiates beyond; some tier jumps are irreversible.

2. What are the four types of billed tokens?

Answer: Input, output, reasoning, and cached tokens (read from cache)—response cost fields detail them, the tokenize endpoint helps forecast.

3. How does the API's automatic cache work?

Answer: Repeated prefixes are served from cache at a reduced rate: keeping prompts stable at the head of the request maximizes cache hit rate—a major cost lever.

4. What do you need to know about model lifecycles?

Answer: Aliases (latest) follow updates, pinned versions guarantee reproducibility—you choose one or the other per use case, and track deprecations.

5. How to handle a migration from a legacy model?

Answer: Inventory usage, switch to the current range while re-testing critical prompts, then monitor costs and quality after the switch—migration is a project, not a sed.

With limits understood, tokens counted, cache exploited, and versions managed: serene operations rest on these four disciplines.