Google Vertex AI

Updated November 2, 2026 — Google Cloud. Google now groups its AI offerings under the Gemini Enterprise Agent Platform (“Agent Platform”); the Model Garden remains part of this ecosystem. Available Mistral models include Mistral Medium 3 (mistral-medium-3), Codestral 2 (codestral-2), Mistral OCR 25.05 (mistral-ocr-2505), and Mistral Small 3.1 (mistral-small-2503). With Mistral SDK v2 and later, import the client via from mistralai.gcp.client import MistralGCP and install it with pip install "mistralai[gcp]". Sources: Mistral AI models (Google Cloud) · Vertex AI (Mistral).

Mistral on Google Cloud

Google Cloud Vertex AI integrates Mistral models into its unified ML platform. Access requires a dedicated client, MistralGCP. Its key distinction lies in authentication: you do not manage a Mistral API key because the client relies on the Google Cloud credentials already configured on your machine or service. This lesson covers configuration, Python code, and the feature Vertex AI highlights most: Fill-in-the-Middle (FIM) with Codestral.

Models Available on Vertex AI

Vertex AI provides access to several models; since the catalog evolves regularly, check the official page for the current list. Mistral Medium 3 (mistral-medium-3) serves as the high-performance generalist model, your default choice for text generation or analysis. Codestral 2 (25.08) (codestral-2) specializes in code and supports FIM, detailed later in this lesson. Mistral OCR (25.05) (mistral-ocr-2505) handles optical character recognition, typically to extract text from batches of scanned invoices before passing it to another model. Finally, Mistral Small 3.1 (25.03) (mistral-small-2503) is lightweight and fast, making it ideal for high-volume processing where latency matters more than response nuance.

Prerequisites

Set up your Google Cloud configuration before writing a single line of Python. You need a Google Cloud project with the Vertex AI API enabled, the gcloud CLI installed and configured, and two IAM roles assigned to your service account:

• Vertex AI User — to call models
• Consumer Procurement Entitlement Manager — to enable Mistral models

The second role is the one missing nine times out of ten: without it, model activation fails even though your calls appear correctly authenticated.

# Install gcloud CLI
curl https://sdk.cloud.google.com | bash

# Log in
gcloud auth application-default login

# Configure the project
gcloud config set project mon-projet-gcp

One step remains that you cannot perform via the command line: you must activate Mistral models in the Vertex AI Model Garden before using them. Go to the Vertex AI portal, open Model Garden, search for Mistral, and enable the desired models.

The MistralGCP Client

Installation requires the gcp extra, which includes Google authentication dependencies.

pip install mistralai[gcp]

Initialization takes no arguments in the standard case:

from mistralai.gcp.client import MistralGCP

# Automatic authentication via google.auth.default()
client = MistralGCP()

The client uses Google Cloud Application Default Credentials (ADC). If you have run gcloud auth application-default login, or if your code runs on a machine with a service account, no further configuration is needed. By default, the selected region is europe-west4; you can enforce a different one. Choose based on your users’ location, as every millisecond of network travel adds to generation time.

client = MistralGCP(region="us-central1")

Chat Completions

The call itself holds no surprises: unlike Azure, the model parameter here uses the model’s actual name.

from mistralai.gcp.client import MistralGCP

client = MistralGCP()

response = client.chat.complete(
    model="mistral-medium-3",
    messages=[
        {"role": "system", "content": "You are a cloud architecture expert."},
        {"role": "user", "content": "What are the advantages of Vertex AI compared to a self-hosted deployment?"}
    ],
    temperature=0.3,
    max_tokens=500
)

print(response.choices[0].message.content)

FIM with Codestral

Fill-in-the-Middle is a capability unique to Codestral. Instead of continuing text, the model fills a gap between a beginning (passed in prompt) and an end (passed in suffix). This difference is decisive for IDE autocompletion, where the cursor almost always sits in the middle of a file: the model sees what follows and can therefore write a function body consistent with the upcoming return statement, or complete a missing block in existing code.

from mistralai.gcp.client import MistralGCP

client = MistralGCP()

response = client.fim.complete(
    model="codestral-2",
    prompt="def count_words(file_path: str) -> int:",
    suffix="return n_words"
)

print(response.choices[0].message.content)
# Generates the function body between the prompt and suffix

The example makes even more sense with a real class. Here, the model receives the full header, already-defined attributes, and the following disconnect() method; it must produce the body of connect() consistent with this environment.

response = client.fim.complete(
    model="codestral-2",
    prompt="""class DatabaseConnection:
    def __init__(self, host: str, port: int, database: str):
        self.host = host
        self.port = port
        self.database = database
        self.connection = None

    def connect(self):""",
    suffix="""
    def disconnect(self):
        if self.connection:
            self.connection.close()
            self.connection = None
"""
)
print(response.choices[0].message.content)

The generated code will reuse self.host, self.port, and self.database, and assign self.connection—something standard completion, blind to the rest of the file, would have no reason to do.

Streaming on Vertex AI

Streaming works as elsewhere, by iterating over returned chunks.

stream = client.chat.stream(
    model="mistral-medium-3",
    messages=[
        {"role": "user", "content": "Generate a Terraform script for a GKE cluster."}
    ]
)

for chunk in stream:
    content = chunk.data.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

Differences from the Direct Mistral API

Key Takeaways

• The MistralGCP client uses native Google Cloud authentication (no Mistral API key)
• The default region is europe-west4—adapt it to your use case
• Codestral on Vertex AI supports FIM for code autocompletion
• Enable models in the Model Garden before using them
• Vertex identifiers are versioned (codestral-2, mistral-medium-3) and evolve with each generation: verify the exact value in the Model Garden before wiring up a deployment
• Billing goes through your Google Cloud account, not Mistral