Visualize embeddings

Objectives

• Reduce high-dimensional vectors for 2D/3D visualization
• Use t-SNE and UMAP for visualization
• Interpret visual clusters

The challenge: 3,072 dimensions

Embeddings live in a 3,072-dimensional space that no one can visualize directly. To inspect them, you must project them into two or three dimensions. This accepts some information loss but preserves the essential structure: points that were close should remain close. This is a powerful diagnostic tool. A split cluster or overlapping themes jump out at a glance, whereas aggregate metrics might miss them entirely.

Dimensionality reduction with t-SNE

t-SNE (t-distributed Stochastic Neighbor Embedding) is the classic method. It finds a 2D layout where each point retains its original neighborhood. The example below deliberately builds a corpus with three distinct themes—AI, cooking, and sports—so you can verify that the projection recovers the expected structure.

from openai import OpenAI
from sklearn.manifold import TSNE
import matplotlib.pyplot as plt
import numpy as np

client = OpenAI()

# Example corpus
textes = [
    # AI Cluster
    "Deep learning uses neural networks",
    "Machine learning transforms data science",
    "Transformers revolutionized NLP",
    "GPT-5 is a large language model",
    # Cooking Cluster
    "Chocolate cake recipe",
    "How to prepare ratatouille",
    "Basics of French pastry",
    "Oven baking techniques",
    # Sports Cluster
    "Paris Marathon attracts thousands of runners",
    "Interval training for running",
    "Soccer rules explained",
    "Sports nutrition and performance",
]

categories = (["AI"] * 4 + ["Cooking"] * 4 + ["Sports"] * 4)

# Generate embeddings
response = client.embeddings.create(
    input=textes,
    model="text-embedding-3-large"
)
embeddings = np.array([d.embedding for d in response.data])

# t-SNE reduction to 2D
tsne = TSNE(n_components=2, random_state=42, perplexity=4)
coords = tsne.fit_transform(embeddings)

# Visualization
couleurs = {"AI": "#2563eb", "Cooking": "#dc2626", "Sports": "#16a34a"}
fig, ax = plt.subplots(figsize=(10, 8))

for cat in couleurs:
    mask = [c == cat for c in categories]
    ax.scatter(
        coords[mask, 0], coords[mask, 1],
        c=couleurs[cat], label=cat, s=100, alpha=0.8
    )
    for i, (x, y) in enumerate(coords[mask]):
        idx = [j for j, m in enumerate(mask) if m][i]
        ax.annotate(
            textes[idx][:30] + "...",
            (x, y), fontsize=7, alpha=0.7
        )

ax.legend()
ax.set_title("t-SNE Visualization of Embeddings")
plt.tight_layout()
plt.savefig("tsne_embeddings.png", dpi=150)
plt.show()

Three parameters drive the result. perplexity balances local versus global structure, with typical values between 5 and 50; here it is set to 4 because the corpus contains only twelve points, and a perplexity higher than the available neighbors would produce an incoherent projection. n_iter sets the number of optimization iterations, with a minimum of 250 and ideally more than 1,000. random_state ensures reproducibility: without it, two successive runs yield different figures, making comparison impossible.

Reduction with UMAP

UMAP (Uniform Manifold Approximation and Projection) is often preferred over t-SNE because it is faster and better preserves global structure. In other words, the distance between two clusters on a UMAP plot means something, whereas with t-SNE it is arbitrary.

import umap

# UMAP reduction to 2D
reducer = umap.UMAP(n_components=2, random_state=42, n_neighbors=5)
coords_umap = reducer.fit_transform(embeddings)

fig, ax = plt.subplots(figsize=(10, 8))

for cat in couleurs:
    mask = [c == cat for c in categories]
    ax.scatter(
        coords_umap[mask, 0], coords_umap[mask, 1],
        c=couleurs[cat], label=cat, s=100, alpha=0.8
    )

ax.legend()
ax.set_title("UMAP Visualization of Embeddings")
plt.tight_layout()
plt.savefig("umap_embeddings.png", dpi=150)
plt.show()

The following table summarizes the trade-offs between the two methods.

Interactive 3D visualization

On a corpus of a few hundred points, a static figure becomes unreadable: labels overlap, and you cannot tell which text corresponds to which point. Plotly solves this by moving information into tooltips and allowing you to rotate the point cloud.

import plotly.express as px
import pandas as pd

# Reduction to 3D
reducer_3d = umap.UMAP(n_components=3, random_state=42, n_neighbors=5)
coords_3d = reducer_3d.fit_transform(embeddings)

df = pd.DataFrame({
    "x": coords_3d[:, 0],
    "y": coords_3d[:, 1],
    "z": coords_3d[:, 2],
    "category": categories,
    "text": [t[:50] for t in textes]
})

fig = px.scatter_3d(
    df, x="x", y="y", z="z",
    color="category",
    hover_data=["text"],
    title="3D Embeddings"
)
fig.write_html("embeddings_3d.html")
fig.show()

The resulting HTML file is ready to share. It is an excellent way to show a business team how their documents are distributed without writing a single line of code.

Visualizing the effect of dimensionality reduction

Lesson 2 stated that truncating a vector to 256 dimensions retains about 88% of its quality. That figure remains abstract until you see it. By generating the same corpus at four different sizes and projecting each version, you observe directly when clusters start to touch. This lets you decide based on evidence rather than a general rule.

dimensions_a_tester = [256, 512, 1536, 3072]

fig, axes = plt.subplots(1, 4, figsize=(24, 5))

for ax, dim in zip(axes, dimensions_a_tester):
    resp = client.embeddings.create(
        input=textes,
        model="text-embedding-3-large",
        dimensions=dim
    )
    embs = np.array([d.embedding for d in resp.data])
    tsne = TSNE(n_components=2, random_state=42, perplexity=4)
    coords = tsne.fit_transform(embs)

    for cat in couleurs:
        mask = [c == cat for c in categories]
        ax.scatter(coords[mask, 0], coords[mask, 1],
                   c=couleurs[cat], label=cat, s=60)
    ax.set_title(f"{dim} dimensions")
    ax.legend(fontsize=7)

plt.tight_layout()
plt.savefig("comparaison_dimensions.png", dpi=150)
plt.show()

A warning before interpreting these figures: projection always distorts. An isolated point on the graph is not necessarily isolated in the original space, and two clusters stuck together in 2D may be perfectly separated in 3,072 dimensions. Visualization helps you formulate a hypothesis, never validate it. Validation relies on the metrics we will cover in Lesson 10.

Summary

• t-SNE and UMAP project embeddings into 2D/3D for visualization
• UMAP is generally preferred for its speed and fidelity
• Plotly enables interactive 3D visualizations
• Visualization helps validate embedding quality and detect anomalies