Mistral on Google Vertex AI

Updated November 2, 2026 — Google Cloud. Google now groups its AI offerings under Gemini Enterprise Agent Platform (“Agent Platform”); the Model Garden remains available within it. The available Mistral models are Mistral Medium 3 (mistral-medium-3), Codestral 2 (codestral-2), Mistral OCR 25.05 (mistral-ocr-2505), and Mistral Small 3.1 (mistral-small-2503). With Mistral SDK v2 and later, import the client via from mistralai.gcp.client import MistralGCP and install it with pip install "mistralai[gcp]". Sources: Mistral AI models (Google Cloud) · Vertex AI (Mistral).

Why Vertex AI?

Google Cloud Vertex AI is Google’s MLOps platform, and it hosts partner models, including those from Mistral. The logic here mirrors Azure: if your infrastructure already relies on Google Cloud—BigQuery for data, Cloud Run or GKE for execution—using Vertex prevents you from introducing another vendor into your chain.

The decisive argument here is authentication. You no longer need to distribute, rotate, or monitor a Mistral API key: your services authenticate using their Google service account, exactly as they already do to access a bucket or a BigQuery table. This also adds centralized billing on your Cloud Billing account and access to Google’s MLOps tools.

Models available on Vertex AI

As of April 2026, Vertex AI provides access to the following models:

Prepare your Google Cloud environment

Configuration happens once and conditions everything else:

1. A Google Cloud project with the Vertex AI API enabled
2. The following IAM roles assigned to your account:
   • Vertex AI User — to invoke models
   • Consumer Procurement Entitlement Manager — to enable partner models
3. The gcloud CLI installed and configured

The second role often surprises people: it has nothing to do with inference; it authorizes enabling partner models in the catalog. Without it, you will see Mistral models in the interface but never be able to enable them, and the returned error does not clearly point to this lack of permissions.

# Install gcloud (if not already done)
# See https://cloud.google.com/sdk/docs/install

# Authenticate
gcloud auth application-default login

# Set the default project
gcloud config set project your-project-id

One step is regularly forgotten: go to the Vertex AI console, find the desired Mistral model in the Model Garden, and click Enable. This activation is required only once per model and per project. A call to a non-enabled model fails even if your IAM roles are correct—this is the first thing to check when facing an unexplained authorization error.

A first call without an API key

Notice first what is missing in the code below: no API key, no secret to load. Mistral SDK V2 provides a dedicated client for Google Cloud, MistralGCP, which relies on google.auth.default(): it picks up the credentials that gcloud auth application-default login deposited locally, or the service account attached to the machine in production.

from mistralai.gcp.client import MistralGCP

# Authentication uses google.auth.default()
# Ensure you have run: gcloud auth application-default login
client = MistralGCP()

response = client.chat.complete(
    model="mistral-small-2503",
    messages=[
        {"role": "user", "content": "Explain the architecture of BigQuery."}
    ]
)

print(response.choices[0].message.content)

By default, the client uses the europe-west4 region, in the Netherlands. To change it, simply pass it to the constructor—keeping in mind that available regions depend on the model and that the Vertex AI documentation is authoritative on this point.

client = MistralGCP(region="us-central1")

Streaming follows the same mechanism seen with the direct API: iterate over the stream, access content via chunk.data, and test before printing since some fragments arrive empty.

stream = client.chat.stream(
    model="mistral-small-2503",
    messages=[
        {"role": "user", "content": "List the serverless services of Google Cloud."}
    ]
)

for chunk in stream:
    content = chunk.data.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

The special case of FIM with Codestral

Codestral on Vertex AI supports Fill-in-the-Middle (FIM), a feature that changes how you approach code completion. Instead of asking “write me a function,” you provide the beginning and the end, and the model produces what is missing between the two. The method also changes: it is no longer chat.complete() but fim.complete(), with a prompt for the upstream part and a suffix for the downstream part.

from mistralai.gcp.client import MistralGCP

client = MistralGCP()

response = client.fim.complete(
    model="codestral-2",
    prompt="def calculate_average(grades: list[float]) -> float:",
    suffix="    return average"
)

print(response.choices[0].message.content)

In this example, the model generates the function body between the signature and the return. It knows that a variable average must exist at the end, since it appears in the suffix: the constraint guides it far better than a natural language instruction. This is exactly what happens when an IDE completes a function in the middle of a file, making FIM particularly effective for completion in an editor, filling in missing code blocks, and generating code that must fit into an existing structure.

The Vertex client in TypeScript

In JavaScript and TypeScript, the Google Cloud client is published in a separate package, @mistralai/mistralai-gcp (npm add @mistralai/mistralai-gcp), and is called MistralGCP since SDK version 2.0 (it was previously called MistralGoogleCloud). The region and project ID are declared at construction, and again, no key appears in the code.

import { MistralGCP } from '@mistralai/mistralai-gcp';

const client = new MistralGCP({
  region: 'europe-west4',
  projectId: process.env.GOOGLE_PROJECT_ID,
});

const response = await client.chat.complete({
  model: 'mistral-small-2503',
  messages: [
    { role: 'user', content: 'How to optimize a BigQuery query?' },
  ],
});

console.log(response.choices?.[0]?.message?.content);

Sources: TypeScript SDK @mistralai/mistralai-gcp · TypeScript SDK release notes 2.0.0

Going to production properly

User authentication via gcloud auth application-default login is perfect for development, but it ties your calls to a person: the day that person leaves the team, the service breaks. In production, use service accounts, whose rights are explicit and auditable. Keep europe-west4 as the default region if European compliance is part of your constraints. Enable Cloud Logging to track your calls and diagnose errors, and configure Cloud Billing budgets: on a platform where inference is billed by usage, a budget alert is your only safeguard against a runaway loop.

Key takeaways

• The MistralGCP client uses native Google Cloud authentication (gcloud auth)
• The default region is europe-west4 — modifiable at initialization
• Codestral on Vertex supports FIM (Fill-in-the-Middle) for code completion
• No Mistral API key is required: authentication goes through Google Cloud IAM
• Enable each model in the Model Garden before using it