Charts and Diagrams

Analyzing Visual Data

Charts, diagrams, and tables are ubiquitous in the professional world: financial reports, dashboards, presentations, scientific publications. Grok Vision lets you automatically extract data and insights from these visuals.

Why This Differs from OCR

Chart analysis goes well beyond simple text extraction. The model must:

• Identify the type of chart (bar, line, pie, scatter plot, etc.)
• Read the axes and their scales
• Extract data values
• Understand legends and map them to data series
• Interpret trends and patterns

Analyzing a Simple Chart

def analyze_chart(image_url):
    prompt = """Analyze this chart and provide:

1. Chart type (bar, line, pie, etc.)
2. Chart title (if visible)
3. X and Y axis labels
4. Extracted data as a table
5. Main observed trend
6. Any annotations or notable points

Present the data in a markdown table."""

    response = client.responses.create(
        model="grok-4.5",
        input=[
            {"type": "input_image", "image_url": image_url, "detail": "high"},
            {"type": "input_text", "text": prompt}
        ],
        store=False
    )
    return response.output_text

The detail: "high" mode is essential for reading precise values on axes and legends.

Extracting Tabular Data

To get data ready for direct use in your code:

def extract_chart_data(image_url):
    prompt = """Extract data from this chart in JSON.
Expected format:
{
  "type": "bar|line|pie|scatter",
  "title": "...",
  "x_axis": {"label": "...", "values": [...]},
  "y_axis": {"label": "...", "unit": "..."},
  "series": [
    {
      "name": "Series 1",
      "values": [10, 20, 30, ...]
    }
  ]
}

Estimate numerical values as precisely as possible
based on the position of points/bars relative
to the tick marks."""

    response = client.responses.create(
        model="grok-4.5",
        input=[
            {"type": "input_image", "image_url": image_url, "detail": "high"},
            {"type": "input_text", "text": prompt}
        ],
        store=False
    )
    return response.output_text

Analyzing Technical Diagrams

Grok also handles architecture diagrams, flowcharts, and flow schemas well:

def analyze_diagram(image_url):
    prompt = """Analyze this technical diagram and describe:

1. Diagram type (architecture, flow, flowchart, UML, etc.)
2. Main components/entities
3. Connections and relationships between components
4. Data or process flow (if applicable)
5. Technologies or tools mentioned

If this is a flow diagram, describe the main path
from end to end."""

    response = client.responses.create(
        model="grok-4.5",
        input=[
            {"type": "input_image", "image_url": image_url, "detail": "high"},
            {"type": "input_text", "text": prompt}
        ],
        store=False
    )
    return response.output_text

Comparing Charts

You can send multiple charts in the same request for comparative analysis:

def compare_charts(url_chart_1, url_chart_2):
    prompt = """Compare these two charts and identify:

1. Common points (measured variables, period, etc.)
2. Differences in trend
3. Significant gaps
4. A 3-4 sentence summary"""

    response = client.responses.create(
        model="grok-4.5",
        input=[
            {"type": "input_text", "text": "Chart 1:"},
            {"type": "input_image", "image_url": url_chart_1, "detail": "high"},
            {"type": "input_text", "text": "Chart 2:"},
            {"type": "input_image", "image_url": url_chart_2, "detail": "high"},
            {"type": "input_text", "text": prompt}
        ],
        store=False
    )
    return response.output_text

Professional Use Case: Automated Reports

Imagine a pipeline that transforms dashboard screenshots into text reports:

def generate_dashboard_report(screenshots):
    report_sections = []

    for i, screenshot in enumerate(screenshots, 1):
        prompt = f"""You are a business analyst. This screenshot
is section {i} of a dashboard. Analyze the displayed metrics
and write a professional summary paragraph.

Include:
- Main KPIs and their values
- Trends (increase, decrease, stability)
- Visible alerts or anomalies
- A recommendation if relevant"""

        response = client.responses.create(
            model="grok-4.5",
            input=[
                {"type": "input_image", "image_url": screenshot, "detail": "high"},
                {"type": "input_text", "text": prompt}
            ],
            store=False
        )
        report_sections.append(response.output_text)

    return "\n\n".join(report_sections)

Limitations of Chart Analysis

• Value precision: Grok estimates values based on visual position. For exact values, prefer source data
• 3D charts: Perspective representations complicate value reading
• Overlapping charts: When multiple series overlap, distinction can be difficult
• Small legends: Legends in very small text may be misread even in high mode

Key Takeaways

• Grok analyzes charts by understanding context, not just text
• Use detail: "high" for precise reading of values and legends
• Request a structured format (JSON, markdown table) to exploit the data
• Multi-image comparison works well for cross-analysis
• Extracted values are visual estimates, not exact data