Analyze Charts with the Vision API
On this page
Read Charts Like an Analyst
Charts—bar, line, pie, area—are everywhere in business reports, dashboards, and scientific publications. They present a specific problem: the information is locked inside an image, often without the source data attached. An annual PDF report might contain thirty curves, and no one has the original Excel file anymore.
Mistral’s Vision API can read these charts, interpret trends, and extract numerical data. This lesson shows how to write prompts that produce actionable results instead of vague commentary, and how to structure the output.
Principles of Chart Analysis
Left to its own devices, the model produces a generic sentence like “this chart shows sales evolution.” An effective prompt forces an ordered process instead: identify the chart type (bar, line, pie, scatter), read the axes and legends with their units and scales, extract values by approximating visual position, and only then interpret trends—rise, fall, seasonality, anomalies. Order matters: an interpretation formed before reading the axes is an invented interpretation.
Analyzing a Bar Chart
The system message is the natural place to encode this method once and for all, regardless of the image being analyzed:
from mistralai.client import Mistral
import os
client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
response = client.chat.complete(
model="mistral-large-latest",
messages=[
{
"role": "system",
"content": """You are an expert data analyst. When analyzing a chart:
1. Identify the chart type and its title
2. Describe the axes (labels, units, scale)
3. List the main values you can read
4. Identify trends, highs/lows, and anomalies
5. Write a 2-3 sentence conclusion"""
},
{
"role": "user",
"content": [
{"type": "text", "text": "Analyze this bar chart of quarterly revenue."},
{"type": "image_url", "image_url": "https://exemple.com/ca-trimestriel.png"}
]
}
]
)
print(response.choices[0].message.content)The system prompt structures the model’s response into logical steps, improving both accuracy and readability. You get a consistent report from one chart to the next, which is essential if these responses need to be processed automatically later.
Extracting Numerical Data
To get usable data instead of a description, the instruction must be explicitly restrictive. The model has a natural tendency to comment; you must forbid it:
response = client.chat.complete(
model="mistral-large-latest",
messages=[
{
"role": "system",
"content": "You are a data extractor. Analyze the chart and return the data as a markdown table. Do not comment, do not describe—only extract the data."
},
{
"role": "user",
"content": [
{"type": "text", "text": "Extract the data from this bar chart into a table."},
{"type": "image_url", "image_url": "https://exemple.com/ventes-mensuelles.png"}
]
}
]
)The model will return something like:
| Month | Sales (k€) |
|----------|------------|
| January | 120 |
| February | 135 |
| March | 148 |This format parses in a few lines of code and imports directly into a spreadsheet or DataFrame.
Analyzing a Line Chart (Trends)
Line charts are particularly suited for trend analysis, and the question can then become a full questionnaire posed in the user message:
prompt_tendance = """Analyze this line chart and answer:
1. What is the general trend (rising, falling, stable)?
2. Are there any inflection points? If so, on what dates?
3. Do you observe seasonality?
4. What is the approximate value at the start and end of the period?
5. What projection do you make for the next quarter?"""
response = client.chat.complete(
model="mistral-large-latest",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": prompt_tendance},
{"type": "image_url", "image_url": "https://exemple.com/evolution-prix.png"}
]
}
]
)The fifth question is different in nature from the first four: it asks for extrapolation, not reading. Treat its answer as a hypothesis to validate, never as a fact.
Analyzing a Pie Chart (Distribution)
For pie charts, the question focuses on proportions and concentration rather than evolution:
prompt_camembert = """Analyze this pie chart:
- List each segment with its approximate percentage
- Identify the dominant segment
- Indicate whether the distribution is concentrated or dispersed
Return the data in a markdown table."""
response = client.chat.complete(
model="mistral-large-latest",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": prompt_camembert},
{"type": "image_url", "image_url": "https://exemple.com/repartition-marche.png"}
]
}
]
)Always verify that the sum of the returned percentages is close to 100: a total of 87% signals an incomplete read, a forgotten segment, or one hidden by the legend.
Report Analysis Pipeline
These isolated calls become useful at the scale of an entire report. The following script scans a folder of exported charts, requests a standardized JSON output each time, and logs everything to a single file:
import base64
from pathlib import Path
from mistralai.client import Mistral
import json
client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
def encoder_image(chemin):
with open(chemin, "rb") as f:
b64 = base64.standard_b64encode(f.read()).decode("utf-8")
return f"data:image/png;base64,{b64}"
def analyser_graphique(chemin_image, question=None):
"""Analyze a chart and return structured data."""
if question is None:
question = "Analyze this chart. Return a JSON with: type (bar/line/pie/other), title, data (array of {label, value}), trend, and summary."
response = client.chat.complete(
model="mistral-large-latest",
messages=[
{
"role": "system",
"content": "Return only valid JSON, no markdown code blocks."
},
{
"role": "user",
"content": [
{"type": "text", "text": question},
{"type": "image_url", "image_url": encoder_image(chemin_image)}
]
}
]
)
return response.choices[0].message.content
# Analyze all charts in a folder
dossier = Path("rapport/graphiques/")
resultats = {}
for fichier in sorted(dossier.glob("*.png")):
print(f"Analyse de {fichier.name}...")
resultats[fichier.name] = analyser_graphique(fichier)
# Save results
with open("rapport/analyse_graphiques.json", "w", encoding="utf-8") as f:
json.dump(resultats, f, ensure_ascii=False, indent=2)The instruction “no markdown code blocks” avoids the classic annoyance of JSON wrapped in triple backticks, which breaks parsing.
Limits of Chart Analysis
Four situations challenge the model, and you must know them before building a process around it. First, precise values: they are estimated from the visual position of bars or points, with a margin of error of 5 to 10%. Second, cluttered charts, where multiple series, legends, and small text blur readability. Third, low resolution, which makes labels blurry or pixelated and unreadable—remember the internal resizing described in Lesson 2. Finally, 3D charts, where perspective mechanically distorts height reading.
The practical conclusion is clear: for precise extraction, prefer exporting source data as CSV when available. The Vision API is ideal for quick analysis and synthesis, not for replacing a spreadsheet. It shines when source data is gone; it is a poor choice when data is within reach.
Key Takeaways
- Use Mistral Large 3 for chart analysis—precision is critical
- Structure your prompt in steps: type, axes, values, trends, conclusion
- Explicitly request an output format (markdown table, JSON) for usable data
- The model estimates values with a 5–10% margin—do not use it for exact figures
- Combine the Vision API with a CSV export for the best of both worlds