Encode and Send Images in Base64

Why Use Base64?

In the previous lesson, you learned how to send images via URL. However, most images processed in business contexts don’t have a URL: an invoice that just arrived by email, a screenshot generated by an automated test, or a photo taken by an operator on a production line. Publishing these files on an internet-accessible server just to analyze them would be absurd, and often prohibited.

This is where Base64 encoding comes in. It transforms the binary data of an image into a text string that you include directly in your API request, without needing an external URL or intermediate hosting.

When to Use URL vs. Base64

The rule is simple: if the image is accessible via a stable URL, use the URL. Otherwise, use Base64.

Encoding an Image to Base64

No dependencies are required; Python provides the base64 module in its standard library:

import base64

def encode_image(file_path):
    """Encode an image file to a Base64 string."""
    with open(file_path, "rb") as f:
        return base64.standard_b64encode(f.read()).decode("utf-8")

# Encode a local image
image_b64 = encode_image("captures/screenshot.png")
print(f"Base64 string length: {len(image_b64)} characters")

The "rb" mode when opening the file is not optional: reading an image in text mode will raise a decoding error. Expect long strings—encoding increases the size by about 33% compared to the original file, so a 1 MB image produces roughly 1.33 MB of text.

Data URI Format

The raw string isn’t enough: the API needs to know the image format. You must therefore present it as a data URI, which prefixes the data with its MIME type:

data:image/{format};base64,{base64_data}

The four usable prefixes correspond to the supported formats:

• data:image/jpeg;base64,...
• data:image/png;base64,...
• data:image/gif;base64,...
• data:image/webp;base64,...

Announcing a PNG as image/jpeg causes a processing error; hence the detection function in the following example.

Complete Example

Here is the full chain, from reading the local file to the model’s response:

import base64
import os
from mistralai.client import Mistral

def encode_image(chemin):
    """Encode a local image to Base64."""
    with open(chemin, "rb") as f:
        return base64.standard_b64encode(f.read()).decode("utf-8")

# Determine MIME type
def obtenir_mime(chemin):
    """Return MIME type based on extension."""
    extensions = {
        ".jpg": "image/jpeg",
        ".jpeg": "image/jpeg",
        ".png": "image/png",
        ".gif": "image/gif",
        ".webp": "image/webp"
    }
    ext = os.path.splitext(chemin)[1].lower()
    return extensions.get(ext, "image/jpeg")

# Prepare the image
chemin_image = "documents/facture-2024.png"
image_b64 = encode_image(chemin_image)
mime_type = obtenir_mime(chemin_image)
data_uri = f"data:{mime_type};base64,{image_b64}"

# Call the API
client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])

response = client.chat.complete(
    model="mistral-small-latest",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Extract all information from this invoice."},
                {"type": "image_url", "image_url": data_uri}
            ]
        }
    ]
)

print(response.choices[0].message.content)

Key takeaway: the field is always called image_url, even when it contains a data URI. The message structure doesn’t change one bit from Lesson 4; only the value differs.

Encoding Images from Other Sources

A file on disk is just one case among many. If your image already exists in memory, as a Pillow object from a capture or processing step, writing it to disk just to read it back immediately would be an unnecessary detour—a memory buffer works fine:

import base64
from io import BytesIO
from PIL import Image

def pil_to_base64(pil_image):
    """Convert a Pillow image to a Base64 data URI."""
    buffer = BytesIO()
    pil_image.save(buffer, format="PNG")
    b64 = base64.standard_b64encode(buffer.getvalue()).decode("utf-8")
    return f"data:image/png;base64,{b64}"

# Example with a Pillow image
img = Image.open("capture.png")
data_uri = pil_to_base64(img)

Same logic applies to an image retrieved via HTTP that you don’t want to store, for example because it comes from a short-lived link:

import base64
import requests

def url_to_base64(url):
    """Download an image and convert it to a data URI."""
    response = requests.get(url, timeout=10)
    response.raise_for_status()
    content_type = response.headers.get("Content-Type", "image/jpeg")
    b64 = base64.standard_b64encode(response.content).decode("utf-8")
    return f"data:{content_type};base64,{b64}"

data_uri = url_to_base64("https://example.com/photo.jpg")

Optimize Size Before Encoding

A smartphone photo today weighs 4 to 8 MB, which becomes nearly 10 MB once encoded. The API will resize it anyway, as you saw in Lesson 2: you’d be paying for the transfer for nothing. So always resize before encoding.

from PIL import Image
from io import BytesIO
import base64

def preparer_et_encoder(chemin, taille_max=1024, qualite=85):
    """Resize, compress, and encode an image."""
    img = Image.open(chemin)

    # Resize if necessary
    if max(img.size) > taille_max:
        img.thumbnail((taille_max, taille_max))

    # Convert to JPEG to reduce size
    buffer = BytesIO()
    img.convert("RGB").save(buffer, format="JPEG", quality=qualite)

    b64 = base64.standard_b64encode(buffer.getvalue()).decode("utf-8")
    taille_ko = len(buffer.getvalue()) / 1024
    print(f"Encoded image: {taille_ko:.0f} KB")

    return f"data:image/jpeg;base64,{b64}"

data_uri = preparer_et_encoder("photo_haute_resolution.png")

Displaying the final size isn’t just decorative: keep it during your tests, as it will immediately tell you if your pipeline is sending 80 KB or 8 MB on every call.

Limits to Know

Three constraints come with Base64. The encoded image adds to the HTTP request body, so beyond 10 MB you risk timeouts. Encoding also loads the entire image into memory: for batch processing, handle images one by one rather than building a list of a thousand encoded strings. Finally, unlike URLs, Base64 images are retransmitted in full with every request—analyzing the same invoice twice means sending it twice.

Key Takeaways

• Base64 is ideal for local or dynamically generated images
• The data URI format combines MIME type and data: data:image/png;base64,...
• Resize and compress your images before encoding to optimize latency
• Base64 increases size by about 33%—keep your images under 10 MB
• Use URL when the image is already hosted, Base64 in all other cases