Data Types and Dimensions

Optimizing Size and Quality

codestral-embed gives you precise control over two axes: the number of dimensions and the data type (dtype) of the values. These parameters let you tune the trade-off between search quality and storage or compute costs.

This becomes critical as volumes grow. An index of two million functions at 1536 float dimensions weighs about twelve gigabytes and no longer fits in memory on a standard machine; the same index in int8 at 768 dimensions drops below one gigabyte and becomes queryable in real time again. It’s this arithmetic, not an aesthetic preference, that should guide your choices.

The Five Output Types

float (default)

The float type produces 32-bit floating-point values. This is the most precise format, ideal when quality takes priority over storage space.

from mistralai.client import Mistral

client = Mistral(api_key=api_key)

response = client.embeddings.create(
    model="codestral-embed",
    output_dtype="float",
    inputs=["def hello(): print('world')"],
)

embedding = response.data[0].embedding
print(f"Type: float, Values: {embedding[:5]}")
# [0.0234375, -0.15625, 0.08984375, 0.01171875, -0.046875]

int8

Each dimension is an integer between -128 and 127. This uses 4x less memory than float32 with minimal quality loss. Compare the displayed values with those in the previous example: it’s the same vector, quantized to a coarser scale, and the ranking of results remains almost unchanged.

response = client.embeddings.create(
    model="codestral-embed",
    output_dtype="int8",
    inputs=["def hello(): print('world')"],
)

embedding = response.data[0].embedding
print(f"Type: int8, Values: {embedding[:5]}")
# [3, -20, 11, 1, -6]

uint8

Similar to int8 but with values between 0 and 255 (unsigned). Useful for certain vector search engines that only accept positive values—a constraint you usually discover during integration, not before.

response = client.embeddings.create(
    model="codestral-embed",
    output_dtype="uint8",
    inputs=["def hello(): print('world')"],
)

embedding = response.data[0].embedding
print(f"Type: uint8, Values: {embedding[:5]}")
# [131, 108, 139, 129, 122]

binary

The most compact format. Each dimension is quantized to 1 bit, then bits are grouped in packs of 8 into int8 values. The resulting vector is 32x smaller than float—the length of 192 displayed below corresponds exactly to the original 1536 dimensions, packed eight by eight.

response = client.embeddings.create(
    model="codestral-embed",
    output_dtype="binary",
    inputs=["def hello(): print('world')"],
)

embedding = response.data[0].embedding
print(f"Type: binary, Length: {len(embedding)}")
# Length: 192 (instead of 1536 in float)

ubinary

Identical to binary but with uint8 values (0-255) instead of int8 (-128 to 127). Choosing between the two depends on what your downstream storage engine accepts, not on search quality.

Impact on Quality

Compression figures don’t tell you what you’re actually losing. The following function measures this on a concrete example: a palindrome test, queried in French, encoded successively with three dtypes. The resulting similarities give you the order of magnitude of degradation on your own data.

import numpy as np
from sklearn.metrics.pairwise import cosine_similarity

def compare_dtypes(code_snippet, query):
    """Compare search quality by dtype."""
    dtypes = ["float", "int8", "uint8"]
    results = {}

    for dtype in dtypes:
        # Encode code and query with the same dtype
        resp_code = client.embeddings.create(
            model="codestral-embed",
            output_dtype=dtype,
            inputs=[code_snippet],
        )
        resp_query = client.embeddings.create(
            model="codestral-embed",
            output_dtype=dtype,
            inputs=[query],
        )

        emb_code = np.array(resp_code.data[0].embedding, dtype=np.float32)
        emb_query = np.array(resp_query.data[0].embedding, dtype=np.float32)

        sim = cosine_similarity([emb_code], [emb_query])[0][0]
        results[dtype] = sim

    return results

# Test
code = "def is_palindrome(s): return s == s[::-1]"
query = "fonction qui vérifie si un mot est un palindrome"

scores = compare_dtypes(code, query)
for dtype, score in scores.items():
    print(f"  {dtype:8s}: similarity = {score:.4f}")

Note that the code and query are always encoded with the same dtype inside the loop. Mixing types between the index and the query would produce inconsistent scores, and the error is all the more insidious because it raises no exception.

Reducing Dimensions

The output_dimension parameter controls the number of dimensions in the vector. The loop below iterates through six values and displays the memory footprint of a single vector for each, first in float then in int8.

import numpy as np

# Compare different dimensions
dims = [256, 512, 768, 1024, 1536, 3072]

for dim in dims:
    response = client.embeddings.create(
        model="codestral-embed",
        output_dimension=dim,
        inputs=["def quicksort(arr): ..."],
    )
    emb = response.data[0].embedding

    # Approximate memory size
    size_float32 = dim * 4  # 4 bytes per float32
    size_int8 = dim * 1     # 1 byte per int8

    print(f"  {dim:5d} dims -> {size_float32:6d} bytes (float) | {size_int8:5d} bytes (int8)")

Output:

    256 dims ->   1024 bytes (float) |   256 bytes (int8)
    512 dims ->   2048 bytes (float) |   512 bytes (int8)
    768 dims ->   3072 bytes (float) |   768 bytes (int8)
   1024 dims ->   4096 bytes (float) |  1024 bytes (int8)
   1536 dims ->   6144 bytes (float) |  1536 bytes (int8)
   3072 dims ->  12288 bytes (float) |  3072 bytes (int8)

Multiply the right column by your actual number of vectors and you immediately get your server sizing. Between 3072 dimensions in float and 768 in int8, the ratio is sixteen: the same index goes from twelve gigabytes to seven hundred fifty megabytes.

Selection Guide

The following configurations cover the most frequent scenarios:

The “real-time” line describes a two-stage architecture worth understanding: the very lightweight binary index serves to retrieve a few hundred candidates in milliseconds; only these candidates are then re-ranked using the float vectors, which are more costly but much finer. You get the speed of binary and, at the top of the list, the precision of float.

Key Takeaways

• codestral-embed supports 5 output types: float, int8, uint8, binary, ubinary
• The binary type is 32x more compact than float, with acceptable quality loss for pre-filtering
• The first dimensions contain the most information (Matryoshka representation)
• Combine reduced dimensions + int8 for very compact indexes
• Always test on your real data before choosing production parameters