Chain of Thought

Updated November 3, 2026 based on the API reference and xAI best practices: max_completion_tokens caps only the visible response, not the reasoning; the recommended timeout is at least 900 seconds. Sources: Chat Completions · Best practices.

Reason step by step

Chain of Thought (CoT) is a technique that involves asking the model to break down its reasoning into explicit steps before giving its final answer. Instead of jumping straight to the conclusion, the model details its thought process, which significantly improves response quality for complex tasks.

This approach works particularly well for logic problems, calculations, multi-criteria analyses, and any task that requires multi-step reasoning.

Why chain of thought works

When you ask Grok to solve a problem directly, it generates the most probable answer without necessarily checking its consistency. By asking it to reason step by step, you prompt it to:

1. Break down the problem into sub-problems
2. Solve each sub-problem sequentially
3. Verify consistency between steps
4. Arrive at a conclusion based on the reasoning

Example without chain of thought

Q: A store sells t-shirts for 25 euros. With a 20% discount and 20% VAT, what is the final price?
A: 24 euros.

Example with chain of thought

Q: A store sells t-shirts for 25 euros. With a 20% discount and 20% VAT, what is the final price? Reason step by step.
A:
1. Base price: 25 euros
2. 20% discount: 25 x 0.20 = 5 euros
3. Price after discount: 25 - 5 = 20 euros
4. 20% VAT: 20 x 0.20 = 4 euros
5. Final price including tax: 20 + 4 = 24 euros
The final price is 24 euros.

The result may be identical, but the explicit reasoning lets you verify each step and catch potential errors.

Grok reasoning models

xAI offers models specifically designed for reasoning. These models natively integrate chain of thought without you having to request it explicitly.

Adjust reasoning effort

Earlier models like grok-3-mini exposed a reasoning_effort parameter (low/high). On the current lineup, reasoning effort is set via the reasoning parameter in the Responses API, shown below:

• low: fast reasoning, fewer tokens consumed, suited for simple questions
• medium: balance between depth of thought and cost, suited for most cases
• high: in-depth reasoning, more tokens, suited for complex problems

grok-4.20-0309-reasoning: advanced reasoning

Current Grok models integrate encrypted reasoning. The thought process is not visible in plain text, but it is more performant. With the Responses API, you can control the effort via the reasoning parameter:

{
  "model": "grok-4.20-0309-reasoning",
  "input": "Analyze the advantages and disadvantages of migrating to a microservices architecture",
  "reasoning": {
    "effort": "high"
  }
}

Available levels are low, medium, and high.

Triggering techniques

Even with non-reasoning models, you can activate chain of thought with simple phrasing in your prompt:

• “Reason step by step before answering”
• “Break down your reasoning”
• “Explain your approach then give your conclusion”
• “Think aloud”

In the system message

{
  "role": "system",
  "content": "You are a financial analyst. For each question, break down your reasoning into numbered steps before giving your final conclusion."
}

Reasoning tokens and costs

Reasoning tokens are counted in usage and billed as output tokens. They appear in the reasoning_tokens field of the response:

{
  "usage": {
    "completion_tokens_details": {
      "reasoning_tokens": 150
    }
  }
}

The max_completion_tokens parameter limits only the visible response: internal reasoning does not count toward it, and its quantity is set by the reasoning effort.

xAI best practices require a timeout of at least 900 seconds per request, and examples in its Reasoning guide go up to 3,600 seconds for complex problems.

Limitations of reasoning models

Certain parameters are not available on reasoning models:

• frequency_penalty and presence_penalty: not supported
• stop: not supported
• Including these parameters triggers an error

Try it out

Test the same question with and without chain of thought:

Without CoT:

A train leaves Paris at 8 AM at 120 km/h. Another leaves Lyon (470 km) at 9 AM at 150 km/h. At what time do they meet?

With CoT:

A train leaves Paris at 8 AM at 120 km/h. Another leaves Lyon (470 km) at 9 AM at 150 km/h. At what time do they meet? Reason step by step and show your calculations.

Compare the responses: the version with chain of thought will be more reliable and verifiable.

Key takeaways

• Chain of thought improves reliability on complex tasks by breaking down reasoning
• Current Grok models integrate encrypted reasoning with three effort levels (low/medium/high)
• Reasoning tokens are billed and count toward the output token limit
• Certain parameters (penalties, stop) are not compatible with reasoning models
• Add “reason step by step” to your prompts to activate CoT on any model