Use Case: Code Embeddings
On this page
Beyond Search
Code embeddings aren’t just for semantic search. They unlock advanced applications: repository analytics, clustering, duplicate detection, and development assistants. In this lesson, you’ll explore these use cases with concrete Python code.
These four use cases share a common trait: they leverage the geometry of the index rather than a query. Once your files are placed in vector space, their relative positions tell a story about the project itself: what resembles what, what repeats, and what stands out.
Repository Analytics
Analyze the semantic structure of a project to understand how the code is organized. K-Means groups files into five families based on their vector proximity, without ever looking at the folder tree. The result is often instructive: on a messy project, you’ll see files from very different directories land in the same cluster because they actually deal with the same subject.
import numpy as np
from sklearn.cluster import KMeans
from mistralai.client import Mistral
client = Mistral(api_key=api_key)
def analyze_repo(code_files, n_clusters=5):
"""Analyze the semantic structure of a repository."""
# Encode all files
inputs = [f["content"][:3000] for f in code_files]
response = client.embeddings.create(
model="codestral-embed",
inputs=inputs,
)
embeddings = np.array([d.embedding for d in response.data])
# K-Means Clustering
kmeans = KMeans(n_clusters=n_clusters, random_state=42, n_init=10)
labels = kmeans.fit_predict(embeddings)
# Group files by cluster
clusters = {}
for i, label in enumerate(labels):
if label not in clusters:
clusters[label] = []
clusters[label].append(code_files[i]["path"])
return clusters
# Usage
clusters = analyze_repo(code_files, n_clusters=5)
for cluster_id, files in clusters.items():
print(f"\nCluster {cluster_id} ({len(files)} files):")
for f in files[:5]:
print(f" - {f}")The random_state=42 fixes the initialization: without it, two successive runs would yield different cluster numbers, making it impossible to compare analyses over time.
Duplicate Detection
Identify duplicated or near-duplicated code in your base. Unlike a classic detection tool that compares text, this approach also spots rewritten duplications—the file copied from one module to another, renamed, with different variables but the same logic.
from sklearn.metrics.pairwise import cosine_similarity
def find_duplicates(code_files, embeddings, threshold=0.95):
"""Find pairs of near-identical files."""
sim_matrix = cosine_similarity(embeddings)
duplicates = []
for i in range(len(code_files)):
for j in range(i + 1, len(code_files)):
if sim_matrix[i][j] >= threshold:
duplicates.append({
"file_a": code_files[i]["path"],
"file_b": code_files[j]["path"],
"similarity": float(sim_matrix[i][j]),
})
# Sort by decreasing similarity
duplicates.sort(key=lambda x: x["similarity"], reverse=True)
return duplicates
# Find duplicates
dupes = find_duplicates(code_files, embeddings, threshold=0.92)
print(f"{len(dupes)} potential duplicate pairs found:")
for d in dupes[:10]:
print(f" {d['similarity']:.3f} : {d['file_a']} <-> {d['file_b']}")The inner loop starts at i + 1 to compare each pair only once. Be mindful of the cost, though: the similarity matrix is quadratic, and beyond a few thousand files you’ll need to switch to an approximate index rather than an exhaustive calculation. Lowering the threshold from 0.95 to 0.92 casts a wider net for reworked duplications, at the cost of a few false positives to sort manually.
Clustering by Functionality
K-Means requires you to know the number of groups in advance, which is rarely the case when discovering a project. DBSCAN addresses this limitation: it determines the number of groups itself and, crucially, allows a file to belong to none—which is the meaning of the “Unclassified” label in the code below.
from sklearn.cluster import DBSCAN
def cluster_by_functionality(code_files, embeddings, eps=0.3, min_samples=2):
"""Group code by functionality using DBSCAN."""
# DBSCAN on cosine distances
distance_matrix = 1 - cosine_similarity(embeddings)
clustering = DBSCAN(eps=eps, min_samples=min_samples, metric="precomputed")
labels = clustering.fit_predict(distance_matrix)
# Organize by cluster
groups = {}
for i, label in enumerate(labels):
group_name = f"Group {label}" if label >= 0 else "Unclassified"
if group_name not in groups:
groups[group_name] = []
groups[group_name].append(code_files[i]["path"])
return groups
groups = cluster_by_functionality(code_files, embeddings)
for name, files in groups.items():
print(f"\n{name} ({len(files)} files):")
for f in files[:3]:
print(f" - {f}")Converting via 1 - cosine_similarity turns similarity into distance, the format DBSCAN expects with metric="precomputed". The parameter eps=0.3 therefore means two files are neighbors starting from a cosine similarity of 0.7.
Contextual Code Assistant
This final example assembles the previous building blocks into a true RAG system for code: searching for relevant files, building a context, then calling the LLM. The 0.5 threshold filters upstream—it’s better to give the model two files truly related to the question than three, one of which is off-topic and will mislead it.
def code_assistant(question, code_files, embeddings, top_k=3):
"""Find relevant code to answer a question."""
# Encode the question
response = client.embeddings.create(
model="codestral-embed",
inputs=[question],
)
query_emb = np.array(response.data[0].embedding)
# Find the most relevant code
sims = cosine_similarity([query_emb], embeddings)[0]
top_indices = np.argsort(sims)[::-1][:top_k]
context_parts = []
for idx in top_indices:
if sims[idx] > 0.5: # Minimum threshold
context_parts.append(
f"# File: {code_files[idx]['path']}\n"
f"{code_files[idx]['content'][:1500]}"
)
context = "\n\n---\n\n".join(context_parts)
# Build the prompt for the LLM
prompt = f"""Here is relevant code from our project:
{context}
Developer question: {question}
Answer based on the code above."""
# Call the LLM with context
chat_response = client.chat.complete(
model="mistral-large-latest",
messages=[{"role": "user", "content": prompt}],
)
return chat_response.choices[0].message.content
# Usage
answer = code_assistant(
"How does authentication work in this project?",
code_files,
embeddings,
)
print(answer)The file path is injected as a comment above each snippet. This isn’t decorative: the model uses it to cite its sources, and the developer reading the response knows immediately where to go verify.
Code Quality Metrics
Finally, embeddings help spot what stands out as unusual. The function calculates the centroid of the index—the project’s “average file”—then measures how far each one deviates. A z-score greater than 2 signals a file that resembles nothing else: sometimes a piece of code generated or copied from outside, sometimes simply the only file written in an old style. In both cases, it’s a good candidate for review.
def detect_outliers(code_files, embeddings, threshold=2.0):
"""Detect atypical code files."""
# Calculate the centroid (mean vector)
centroid = embeddings.mean(axis=0)
# Distance of each file to the centroid
distances = np.linalg.norm(embeddings - centroid, axis=1)
# Files beyond N standard deviations
mean_dist = distances.mean()
std_dist = distances.std()
outliers = []
for i, dist in enumerate(distances):
z_score = (dist - mean_dist) / std_dist
if z_score > threshold:
outliers.append({
"path": code_files[i]["path"],
"z_score": float(z_score),
})
outliers.sort(key=lambda x: x["z_score"], reverse=True)
return outliers
outliers = detect_outliers(code_files, embeddings)
print(f"{len(outliers)} atypical files:")
for o in outliers:
print(f" z={o['z_score']:.2f} : {o['path']}")Remember that “atypical” doesn’t mean “bad.” The tool gives you a list of files to look at, not a verdict: it’s a starting point for human review, never a metric to display on a quality dashboard.
Key Takeaways
- Code embeddings enable automatic clustering by functionality
- Duplicate detection uses a cosine similarity threshold (0.92–0.95)
- A code assistant combines semantic search with an LLM for code RAG
- Atypical files (outliers) can signal code needing review
- DBSCAN is preferable to K-Means when the number of groups is unknown