Analyze Images Found by Web Search

Go beyond text with image analysis

Web search isn’t limited to text. When you enable the enable_image_understanding parameter, the Grok model can analyze images found during its searches. This opens up new possibilities: interpreting charts, reading infographics, understanding screenshots, or analyzing news photos.

This capability is particularly useful for use cases where visual information complements or replaces text, such as analyzing trends from stock charts, understanding technical diagrams, or extracting data from tables published as images.

Enable image understanding

You enable this via the enable_image_understanding parameter in the web_search tool configuration:

response = client.responses.create(
    model="grok-4.5",
    input="Show me the evolution of Tesla's stock price this week",
    tools=[{
        "type": "web_search",
        "enable_image_understanding": True
    }]
)

When this parameter is enabled, the model has access to an additional internal tool named view_image. It can invoke this tool to examine images encountered during its search and extract relevant information.

How it works

The process happens in several phases:

1. The model runs a standard web search
2. Among the results, it identifies potentially relevant images
3. It uses the view_image tool to analyze these images
4. It integrates the visual information into its final response

You don’t need to handle any additional interaction. The model orchestrates everything autonomously.

Combine domain filtering and image analysis

You can enable image analysis at the same time as domain filtering:

response = client.responses.create(
    model="grok-4.5",
    input="Visual summary of the latest publications on arxiv in computer vision",
    tools=[{
        "type": "web_search",
        "allowed_domains": ["arxiv.org"],
        "enable_image_understanding": True
    }]
)

Relevant use cases

Brand visual monitoring

Monitor how your brand appears visually online: logos used, screenshots of your product, infographics mentioning your company.

response = client.responses.create(
    model="grok-4.5",
    input="How is the Corsen AI brand represented visually online?",
    tools=[{
        "type": "web_search",
        "enable_image_understanding": True
    }]
)

Visual data analysis

Interpret charts and tables published in online reports without having to download them manually:

response = client.responses.create(
    model="grok-4.5",
    input="Analyze the European AI market growth charts published recently",
    tools=[{
        "type": "web_search",
        "allowed_domains": ["statista.com", "mckinsey.com"],
        "enable_image_understanding": True
    }]
)

Limitations to know

Image analysis increases response time because the model must process visual data in addition to text. It also consumes more output tokens. Enable it only when visual information adds real value to the response.

The quality of the analysis also depends on the resolution and clarity of the images found. High-resolution charts with legible labels yield better results than blurry or cluttered images.

Hands-on practice

Create a script that asks Grok to analyze visual trends in a sector (for example, performance charts for a stock index or infographics from an industry report). Compare the richness of the response obtained with and without enable_image_understanding.

Key takeaways

• The enable_image_understanding parameter allows the model to analyze images from web results
• The model uses the view_image tool internally to examine relevant visuals
• This parameter can be combined with allowed_domains or excluded_domains
• Image analysis increases processing time and token consumption
• This feature is ideal for visual monitoring, chart analysis, and extracting data from infographics