Temperature, top_p, and Generation Parameters

Updated November 3, 2026 based on OpenAI documentation: temperature and top_p are only accepted with a reasoning effort of none, whereas GPT-5.6 models reason by default (medium). Calls that set these parameters must therefore specify reasoning={"effort": "none"} (Responses API) or reasoning_effort="none" (Chat Completions). Sources: Using GPT-6 · GPT-5.6 Terra.

Temperature, top_p, and Generation Parameters

When you send a prompt to a language model, the generated text is not deterministic by default: two strictly identical calls can return two different responses. Generation parameters let you control the degree of creativity or rigor in those responses. Mastering them serves two concrete goals: obtaining results adapted to each use case, and knowing where to look when an integration test suite starts failing one out of three times with no apparent explanation.

How the Model Chooses the Next Token

At each step of generation, the model calculates a probability distribution over its entire vocabulary, then samples the next token from that distribution. The “best” word does not mechanically pop out; it is a draw. Generation parameters intervene before sampling, either tightening the distribution around the most likely candidates or flattening it to give outsiders a chance. The two calls below illustrate the extremes of this need: a bakery slogan is only valuable if it surprises, whereas the syntax of a list comprehension must never surprise.

from openai import OpenAI

client = OpenAI()

# Creative response (high temperature)
response_creative = client.responses.create(
    model="gpt-5.6-terra",
    reasoning={"effort": "none"},  # temperature requires effort none
    instructions="You are a creative writer.",
    input="Propose a slogan for an artisanal bakery.",
    temperature=1.2
)

# Factual response (low temperature)
response_factual = client.responses.create(
    model="gpt-5.6-terra",
    reasoning={"effort": "none"},  # temperature requires effort none
    instructions="You are a precise technical assistant.",
    input="What is the syntax of a list comprehension in Python?",
    temperature=0.1
)

Temperature

Temperature is a multiplicative factor applied to logits before the softmax. At 0, the distribution tightens to the point that the model always selects the most probable token: behavior becomes quasi-deterministic, which is exactly what you want from a data extractor replayed in a unit test. Between 0.1 and 0.3, responses remain very consistent from call to call: this is the territory of code and factual answers. The 0.7 to 0.9 range balances creativity and consistency, suiting general writing. Beyond that, between 1.0 and 1.5, outputs become varied and sometimes disorienting—the desired effect for brainstorming, but a fatal flaw everywhere else.

The table below gathers settings that work well for the most frequent use cases. Treat it as a starting point to validate on your own prompts: a highly constrained prompt tolerates a higher temperature than you might think, while a loose prompt derails sooner.

Top_p (Nucleus Sampling)

The top_p parameter acts on the same distribution, but through a completely different mechanism: it filters tokens based on their cumulative probability. With top_p=0.9, the model considers only the tokens whose cumulative probability reaches 90%, purely and simply removing the long tail of improbable choices. Where temperature distorts the probabilities of all candidates, top_p truncates the list of admissible candidates.

response = client.responses.create(
    model="gpt-5.6-terra",
    reasoning={"effort": "none"},  # temperature requires effort none
    instructions="Generate a technical summary.",
    input="Summarize how TLS 1.3 works.",
    temperature=0.3,
    top_p=0.95
)

This leads to a practical rule you won’t regret applying: do not touch temperature and top_p at the same time. Since both act on the same mechanism, you would be unable to attribute the observed effect to one or the other, and your settings would become a matter of superstition. Adjust one parameter, leaving the other at its default value; OpenAI recommends starting with temperature.

Other Parameters That Matter

max_output_tokens limits the response length in tokens. This is your safeguard for both costs and endless responses, particularly useful when the output must fit within a constrained interface field: a response framed during generation is better than truncation on the display side.

response = client.responses.create(
    model="gpt-5.6-terra",
    instructions="Answer in one sentence.",
    input="What is DNS?",
    max_output_tokens=100
)

Since GPT-5.6, reasoning is no longer a separate model: it is a setting that controls reflection time across the entire family.

response = client.responses.create(
    model="gpt-5.6-terra",
    input="Solve this differential equation: y' + 2y = e^(-x)",
    reasoning={
        "effort": "high"  # "none", "low", "medium", "high", "xhigh", "max"
    }
)

The low level responds quickly to simple questions, medium constitutes an honest compromise for the majority of tasks, and high engages in-depth reasoning that should be reserved for genuinely difficult problems. The differential equation above deserves it; rephrasing a welcome sentence does not—you would be paying for reflection time on a task that requires none.

Observe the Variation Yourself

Nothing replaces seeing these settings in action. Write a Python script that sends the same prompt to GPT-5.6 Terra with five temperature values (0, 0.3, 0.7, 1.0, and 1.3) and displays the five outputs side by side. Choose a frankly creative instruction, for example, “Invent a name for an AI startup”: the near word-for-word repetition at 0 and the dispersion at 1.3 become impossible to miss, and this visual memory will serve as a reference point far longer than a table of values.

Key Takeaways

• Temperature controls the spread of the probability distribution
• top_p filters tokens by cumulative probability
• Never modify both simultaneously
• Low temperature for factual tasks, high for creative ones
• The reasoning setting (from none to max) applies to the entire GPT-5.6 family