Fill-In-The-Middle (FIM)

The Fill-In-The-Middle Principle

Fill-In-The-Middle, or FIM, is Codestral’s most distinctive capability. In standard generation, a model extends text from a starting point with nothing else to consider. FIM changes the game: it lets you fill a gap in the middle of code. You provide the beginning, called the prompt, and the end, called the suffix, and Codestral generates the intermediate code that cleanly connects the two.

This mechanic solves a very concrete problem in IDE autocompletion. When you type in the middle of a file, a relevant suggestion depends as much on what follows the cursor as on what precedes it. A method inserted between two others must respect their style, naming conventions, and the class’s implicit contract. A model that sees only the upstream context proposes plausible code that is often redundant with what already exists ten lines down.

The operation boils down to three elements. The prompt contains the code before the insertion point—in other words, what you have already written before the cursor. The suffix contains the code after the cursor. The completion is the fragment generated by Codestral, which slots in between the two. The final result always obeys the same equation: prompt + completion + suffix = complete code.

The /v1/fim/completions Endpoint

Mistral AI exposes FIM via a dedicated endpoint. Unlike the standard chat endpoint, this one accepts prompt and suffix parameters instead of a message array.

Request Structure

from mistralai.client import Mistral

client = Mistral(api_key="your_api_key")

response = client.fim.complete(
    model="codestral-latest",
    prompt="def fibonacci(n: int):",
    suffix="n = int(input('Enter a number: '))\nprint(fibonacci(n))",
    temperature=0,
)

print(response.choices[0].message.content)

Here, Codestral receives the fibonacci function signature as the prompt and the calling code as the suffix. Its task is to generate the body that turns this signature into a working implementation, knowing that the function will be called with an integer entered via keyboard and that its return value will be printed as is.

Key Parameters

• model — always "codestral-latest" for FIM
• prompt — the code before the cursor (required)
• suffix — the code after the cursor (optional but recommended)
• temperature — 0 for deterministic results, 0.2-0.7 for variety
• max_tokens — limits the length of the generated completion
• stop — list of stop tokens (e.g., ["\n\n"] to stop after a block)

Concrete Example: Completing a Python Class

Let’s look at a common scenario: you are writing a service and are missing a method in the middle of the class.

prompt = """class UserService:
    def __init__(self, db):
        self.db = db

    def get_user(self, user_id: int):
"""

suffix = """
    def delete_user(self, user_id: int):
        return self.db.execute("DELETE FROM users WHERE id = ?", (user_id,))
"""

response = client.fim.complete(
    model="codestral-latest",
    prompt=prompt,
    suffix=suffix,
    temperature=0,
    max_tokens=200,
)

The suffix does all the heavy lifting regarding context. By seeing delete_user, Codestral learns three things: the class queries the database via self.db.execute, it uses parameterized queries with ?, and it returns the result directly without an intermediate layer. It deduces that get_user must follow exactly the same pattern and generates:

        return self.db.execute("SELECT * FROM users WHERE id = ?", (user_id,))

Try the exercise again without the suffix, and you will see the difference. Deprived of this anchor, the model will often invent an ORM or a cursor that does not exist anywhere in your project.

FIM vs. Chat: Choosing the Right Tool

Getting the Most Out of FIM

Completion quality depends first on the context you provide. Include imports, the parent class, and neighboring functions in the prompt. These tell the model which libraries are available and what style to adopt. The suffix deserves special attention because without it, FIM degrades into standard generation and loses its value.

Regarding settings, IDE autocompletion works best at temperature=0 so the same situation always produces the same suggestion. A developer who sees the proposal change with every keystroke will eventually disable it. Keep max_tokens low, between 50 and 200, since you are completing a line or a block, not an entire module. Finally, stop tokens prevent overflow: stop=["\n\n"] cuts generation at the end of the current block rather than letting the model rewrite half the file.

Key Takeaways

• FIM completes code between a prompt (before) and a suffix (after)
• The dedicated endpoint is /v1/fim/completions
• The result is: prompt + completion + suffix = complete code
• The suffix is essential for high-quality contextual completions
• Use temperature=0 and token limits for IDE autocompletion