Analyze an image with Grok

Computer vision with the Grok API

Grok models with vision support can analyze images as well as text. You send an image (by URL or in base64) along with a question, and the model describes it, analyzes it, or extracts visual information. That capability opens up concrete possibilities: extracting data from screenshots, analyzing charts, reading scanned documents.

Image input format

To send an image, you use an array of mixed content (text and image) in the input field:

[
  {
    "type": "input_image",
    "image_url": "https://exemple.com/mon-image.jpg"
  },
  {
    "type": "input_text",
    "text": "What do you see in this image?"
  }
]

The input_image type takes a public URL to the image. The input_text type contains your question or instruction.

Python example

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.getenv("XAI_API_KEY"),
    base_url="https://api.x.ai/v1"
)

response = client.responses.create(
    model="grok-4.5",
    input=[
        {
            "type": "input_image",
            "image_url": "https://exemple.com/graphique-ventes.png"
        },
        {
            "type": "input_text",
            "text": "Analyze this chart. What's the main trend?"
        }
    ]
)

print(response.output[0].content[0].text)

With the chat/completions format

If you use the legacy /v1/chat/completions endpoint, the format is slightly different:

response = client.chat.completions.create(
    model="grok-4.5",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image_url",
                    "image_url": {"url": "https://exemple.com/photo.jpg"}
                },
                {
                    "type": "text",
                    "text": "Describe this image in detail."
                }
            ]
        }
    ]
)

Format differences

Send an image in base64

If your image isn’t reachable by URL (a local file, a generated image), encode it in base64:

import base64

with open("capture.png", "rb") as f:
    image_base64 = base64.b64encode(f.read()).decode("utf-8")

response = client.responses.create(
    model="grok-4.5",
    input=[
        {
            "type": "input_image",
            "image_url": f"data:image/png;base64,{image_base64}"
        },
        {
            "type": "input_text",
            "text": "Read the text in this screenshot."
        }
    ]
)

Real-world use cases

Data extraction

Send a photo of an invoice or a receipt and ask the model to extract the amounts, dates, and reference numbers.

Chart analysis

Submit a chart or a diagram and ask for an interpretation of the trends and key figures.

Coding help

Share a screenshot of an error in your IDE and ask for an explanation and a fix.

Interface description

Send a mockup or a screenshot and ask the model to describe the interface elements it contains.

Vision-capable models

Not every Grok model supports vision. Use grok-4.5, which includes image support. “Mini” or “fast” models may have limited or no support.

Key takeaways

• Use the input_image type with the /v1/responses endpoint to send images
• You can send images by public URL or base64-encoded
• The grok-4.5 model supports image analysis
• Combine image and text in an array to ask questions about visual content