Temperature and Generation Parameters

Lesson rewritten on November 3, 2026 based on xAI best practices, the API reference (Chat Completions and Responses), and the Reasoning guide: xAI recommends temperature at 0.7 and top_p at 0.95 for its models, temperatures above 1 are capped at 1, top_p defaults to 0.95, max_completion_tokens caps only the visible response (128,000 tokens by default), and seed does not guarantee determinism. The previous version recommended temperatures up to 2, a default top_p of 1, and a cap including reasoning. Sources: Best practices · Chat Completions · Reasoning.

Control generation with parameters

Beyond your prompt text, the Grok API accepts several numeric parameters that influence how the model generates its response. With current models, the main lesson is counter-intuitive: handle these settings with significant restraint, and remember that the prompt, far more than the sliders, drives response quality.

Temperature

Temperature controls the degree of randomness in token selection. The API reference describes it classically: a low value, like 0.2, makes the output more focused and deterministic; a higher value, like 0.8, makes it more varied. It accepts values from 0 to 2, but anything above 1 is sampled at 1.

xAI best practices go further: set temperature to 0.7 and do not deviate much from it. According to xAI, Grok models react poorly to other sampling settings, and quality generally drops when you modify them.

{
  "model": "grok-4.7",
  "messages": [{"role": "user", "content": "Propose a slogan for an organic bakery"}],
  "temperature": 0.7,
  "top_p": 0.95
}

To get more creativity, rigor, or conciseness, use the prompt instead: explicitly ask for unexpected proposals, a factual tone, or a response in three bullets. This is the focus of the following lessons.

The top_p parameter (nucleus sampling)

The top_p parameter limits the choice to tokens that cumulate a given probability mass: at 0.1, only tokens representing the top 10% probability are considered. Current Grok text models have this defaulted to 0.95, and values above the model default are capped at that default.

The API reference repeats the usual advice to modify either temperature or top_p, but not both. xAI best practices, newer and specific to its models, ask instead to set both explicitly on every request — 0.7 and 0.95 — so your results do not depend on server defaults, and to use the same values for all models you compare.

max_completion_tokens

This parameter limits the number of tokens in the response. It serves to control the length and cost of the visible part.

{
  "model": "grok-4.7",
  "messages": [{"role": "user", "content": "Summarize this document in three points"}],
  "max_completion_tokens": 500
}

Important points, according to the API reference:

• It applies only to visible tokens: reasoning and function calls do not count; reasoning effort controls reflection
• Without a value, it defaults to 128,000 tokens; the old max_tokens is deprecated
• If the limit is reached, the response is cut off and finish_reason is "length" instead of "stop" (the Responses API names this cap max_output_tokens and signals an incomplete status)

The seed parameter

The seed parameter requests system sampling as deterministic as possible: repeated requests with the same seed and parameters should yield the same result. The API reference specifies that determinism is not guaranteed, and refers to the system_fingerprint field in the response to spot server-side changes.

{
  "model": "grok-4.7",
  "messages": [{"role": "user", "content": "Generate 5 tech startup names"}],
  "temperature": 0.7,
  "top_p": 0.95,
  "seed": 42
}

Use cases for seed:

• Tests and debugging: verify that a prompt produces stable results
• Comparison: test different prompt formulations with the same seed to isolate the effect of the change
• Tracking: log seed and system_fingerprint with each result to explain variance

Parameters rejected by reasoning models

Current Grok models reason: presence_penalty, frequency_penalty, and stop cannot be sent to them, and a request containing them returns an error. logprobs, meanwhile, is ignored from Grok 4.20. If you are porting code written for another provider, remove these parameters.

Recommended settings

xAI does not propose a set of settings per use case: its recommendation is the same everywhere.

{"temperature": 0.7, "top_p": 0.95}

Add a max_completion_tokens adapted to the expected length as needed, a seed for your tests, and, on grok-4.5 to 4.7, a reasoning effort (reasoning_effort) chosen according to task difficulty: this, not temperature, arbitrates speed, cost, and depth.

Hands-on practice

Send the same prompt three times, with temperature at 0.7 and top_p at 0.95, then three times a version of the prompt that specifies the expected tone and format:

Prompt A: "Propose three names for a meditation app"
Prompt B: "Propose three short, evocative, and unexpected names for a meditation app
designed for busy executives. One name per line, no explanation."

Compare the variety and relevance of the responses: the difference comes from the prompt, not the parameters.

Key takeaways

• xAI recommends temperature at 0.7 and top_p at 0.95, set explicitly on every request
• Above 1, temperature is capped at 1; top_p does not exceed the model default (0.95)
• max_completion_tokens limits only the visible response, 128,000 tokens by default
• seed aims for reproducibility without guarantee: track system_fingerprint
• presence_penalty, frequency_penalty, and stop are rejected by reasoning models