Chat Completions and Responses in Batches

Updated November 3, 2026. Based on xAI best practices: temperature recommended at 0.7 and top_p at 0.95 rather than a low temperature. Source: Best practices.

Updated November 2, 2026. Based on xAI documentation: the batch discount is 20% on eligible text models (not 50%), examples follow the REST API and official xai-sdk (client.batch.*; the xAI Batch API does not use the OpenAI format, the OpenAI SDK does not apply), and the Responses API takes precedence over Chat Completions, a legacy endpoint at xAI. Sources: Batch API · REST Reference · Batch Pricing.

The two text endpoints in batch mode

The /v1/responses and /v1/chat/completions endpoints handle most text batches. On eligible models, both benefit from the 20% discount. But xAI does not rank them equally: Responses is the recommended API, Chat Completions a legacy endpoint maintained for existing code. This lesson details their specifics in a batch context.

Responses in a batch

In batch mode, the Responses API accepts the same parameters as in synchronous mode; only the JSONL envelope changes:

{"custom_id": "sum-1", "method": "POST", "url": "/v1/responses", "body": {"model": "grok-4.3", "instructions": "You are a summarization expert.", "input": "Summarize this financial report in five key points: ...", "temperature": 0.3, "max_output_tokens": 500}}

It provides access to Grok’s server tools—web search, X search, code execution, MCP—executed during batch processing:

{"custom_id": "research-1", "method": "POST", "url": "/v1/responses", "body": {"model": "grok-4.3", "input": [{"role": "user", "content": "What are the latest trends in generative AI?"}], "tools": [{"type": "web_search"}]}}

Typical use cases

• Text classification (sentiment, category, urgency)
• Structured information extraction (entities, dates, amounts)
• Summaries, rephrasing, and translations at scale
• Data enrichment via web search

Chat Completions in a batch

Chat Completions remains accepted, with its messages format:

{"custom_id": "sum-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "grok-4.3", "messages": [{"role": "system", "content": "You are a summarization expert."}, {"role": "user", "content": "Summarize this financial report in five key points: ..."}], "temperature": 0.3, "max_completion_tokens": 500}}

The length limit is named max_completion_tokens; the old max_tokens is deprecated. Reserve this endpoint for requests from code already written for it: new features arrive first on Responses.

Differences in batch

Common advanced parameters

temperature ranges from 0 to 2 at xAI, with values beyond 1 sampled at 1. However, xAI best practices recommend setting it to 0.7, with top_p at 0.95, and not deviating much from this: batch consistency comes first from precise instructions and structured outputs.

The output limit (max_output_tokens or max_completion_tokens) must be set explicitly: without it, a request can produce a very long response and consume tokens unnecessarily. It applies only to visible text, not reasoning tokens.

Structured outputs impose a JSON schema on the response, making results programmatically usable without fragile parsing. With Responses, the schema goes in text.format:

{"custom_id": "classify-1", "method": "POST", "url": "/v1/responses", "body": {"model": "grok-4.3", "input": "Classify this comment: « Delivery fast, product as described. »", "text": {"format": {"type": "json_schema", "name": "classification", "schema": {"type": "object", "properties": {"sentiment": {"type": "string", "enum": ["positive", "neutral", "negative"]}}, "required": ["sentiment"], "additionalProperties": false}, "strict": true}}}}

With Chat Completions, the equivalent is response_format, of type json_schema (or json_object for valid JSON without an imposed schema).

Mixing the two endpoints

You can combine Responses and Chat Completions in the same batch, with each line designating its endpoint in the url field:

{"custom_id": "simple-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "grok-4.3", "messages": [{"role": "user", "content": "Summarize in one sentence: ..."}]}}
{"custom_id": "search-1", "method": "POST", "url": "/v1/responses", "body": {"model": "grok-4.3", "input": "AI news this week", "tools": [{"type": "web_search"}]}}

This flexibility allows old Chat Completions requests and new Responses requests to coexist during a migration.

Key takeaways

• Responses is the recommended endpoint, Chat Completions a legacy endpoint; both benefit from the 20% discount on eligible models
• Responses is required for server tools (web, X, code, MCP)
• Set temperature to 0.7 and top_p to 0.95, as xAI recommends, and an appropriate output limit
• Structured outputs go through text.format (Responses) or response_format (Chat Completions)
• Both endpoints can coexist in the same batch