Clustering: Automatic Grouping
On this page
Objectives
- Group texts by theme without predefined labels
- Use K-Means and HDBSCAN on embeddings
- Automatically name clusters
Why clustering?
The classification lesson assumed you already knew which categories existed. Clustering addresses the opposite situation, which is also the most common when exploring a corpus: no one knows what it contains. You retrieve eight thousand verbatims from a satisfaction survey, or six months of support tickets, and the first question isn’t “how many bugs” but “what are people actually talking about.” Clustering discovers thematic groups without being told anything in advance.
Three use cases come up constantly: analyzing thousands of customer feedback items to extract dominant patterns, identifying recurring topics in support tickets to populate a knowledge base, and exploring an unknown document corpus before deciding how to index it.
K-Means on embeddings
K-Means is the go-to algorithm: you give it a number of groups, it places that many centers, and iteratively moves each center toward the average of the points it attracts. Applied to embeddings rather than a bag of words, it groups by meaning: “the app crashes on startup” and “can’t log in since yesterday” end up together even though they share no words.
The example corpus below intentionally contains four families—bugs, feature requests, questions, and compliments—so you can visually verify the resulting grouping. The random_state=42 setting guarantees you’ll get the same clusters from one run to the next, which is essential whenever you need to compare two configurations.
from openai import OpenAI
from sklearn.cluster import KMeans
import numpy as np
client = OpenAI()
# Example corpus
textes = [
"L'application crash au démarrage sur Android",
"Bug : impossible de se connecter depuis hier",
"Le bouton de validation ne fonctionne pas",
"Pourriez-vous ajouter un export Excel ?",
"Suggestion : un mode hors-ligne serait utile",
"Feature request : intégrer Slack",
"Comment changer mon adresse email ?",
"Où trouver ma facture de janvier ?",
"Quel est le prix de l'abonnement annuel ?",
"Votre produit est excellent, bravo !",
"Merci pour le support réactif",
"Je recommande vivement cette application",
]
# Generate embeddings
response = client.embeddings.create(
input=textes,
model="text-embedding-3-large"
)
embeddings = np.array(
[d.embedding for d in sorted(response.data, key=lambda x: x.index)]
)
# K-Means clustering
n_clusters = 4
kmeans = KMeans(n_clusters=n_clusters, random_state=42, n_init=10)
labels = kmeans.fit_predict(embeddings)
# Display clusters
for cluster_id in range(n_clusters):
print(f"\n--- Cluster {cluster_id} ---")
indices = [i for i, l in enumerate(labels) if l == cluster_id]
for idx in indices:
print(f" {textes[idx]}")Finding the right number of clusters
With twelve prepared sentences, choosing four was obvious. With eight thousand verbatims, no one knows the right value. Two metrics help you guess. Inertia, meaning the residual dispersion within groups, mechanically decreases as you add clusters: you’re looking for the “elbow,” the point beyond which it no longer drops significantly. The silhouette score is more decisive because it measures something else: how much closer each point is to its own group than to the nearest neighboring group. Unlike inertia, it has a maximum, and that maximum directly points to a candidate value.
from sklearn.metrics import silhouette_score
import matplotlib.pyplot as plt
inertias = []
silhouettes = []
K_range = range(2, 10)
for k in K_range:
km = KMeans(n_clusters=k, random_state=42, n_init=10)
km.fit(embeddings)
inertias.append(km.inertia_)
silhouettes.append(silhouette_score(embeddings, km.labels_))
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 5))
ax1.plot(K_range, inertias, "bo-")
ax1.set_xlabel("Number of clusters")
ax1.set_ylabel("Inertia")
ax1.set_title("Elbow method")
ax2.plot(K_range, silhouettes, "ro-")
ax2.set_xlabel("Number of clusters")
ax2.set_ylabel("Silhouette score")
ax2.set_title("Silhouette score")
plt.tight_layout()
plt.savefig("choix_k.png", dpi=150)
plt.show()
meilleur_k = K_range[np.argmax(silhouettes)]
print(f"Best k according to silhouette: {meilleur_k}")HDBSCAN: Clustering without K
K-Means suffers from a structural flaw on real-world data: it must assign every point to a cluster. An isolated verbatim that resembles nothing else ends up forcibly attached to the least distant group, where it pollutes interpretation. HDBSCAN reasons in terms of density and accepts saying no: it determines the number of groups itself and places isolated points in a separate category labeled -1.
These outliers aren’t trash. In a ticket corpus, they’re often rare incidents, out-of-scope requests, or weak signals—exactly what deserves human review. The code below displays them separately rather than discarding them.
import hdbscan
clusterer = hdbscan.HDBSCAN(
min_cluster_size=3,
min_samples=2,
metric="euclidean"
)
labels_hdbscan = clusterer.fit_predict(embeddings)
n_clusters = len(set(labels_hdbscan)) - (1 if -1 in labels_hdbscan else 0)
n_outliers = list(labels_hdbscan).count(-1)
print(f"Clusters found: {n_clusters}")
print(f"Outliers: {n_outliers}")
for cluster_id in range(n_clusters):
print(f"\n--- Cluster {cluster_id} ---")
indices = [i for i, l in enumerate(labels_hdbscan) if l == cluster_id]
for idx in indices:
print(f" {textes[idx]}")
if n_outliers > 0:
print("\n--- Outliers ---")
for i, l in enumerate(labels_hdbscan):
if l == -1:
print(f" {textes[i]}")Automatically naming clusters
A clustering algorithm gives you numbers. “Cluster 2” doesn’t belong in a report or a dashboard. The step that makes the result usable consists of submitting a dozen representative members to the LLM and asking it for a short name: “Connection Issues,” “Integration Requests,” “Billing Questions.” Limiting to ten examples controls cost, setting temperature to zero ensures the same cluster always gets the same name, and the loop ignores the -1 label since outliers, by definition, have no common theme.
def nommer_clusters(
textes: list[str],
labels: list[int]
) -> dict[int, str]:
"""Names each cluster using an LLM."""
noms = {}
clusters = set(l for l in labels if l >= 0)
for cluster_id in clusters:
exemples = [
textes[i] for i, l in enumerate(labels)
if l == cluster_id
][:10] # Max 10 examples
response = client.chat.completions.create(
model="gpt-5.6-terra",
messages=[{
"role": "user",
"content": (
f"Voici des textes regroupés dans un même cluster :\n\n"
+ "\n".join(f"- {e}" for e in exemples) +
f"\n\nDonne un nom court (2-4 mots) qui décrit "
f"le thème commun de ce groupe. "
f"Retourne uniquement le nom."
)
}],
temperature=0,
max_tokens=20
)
noms[cluster_id] = response.choices[0].message.content.strip()
return noms
# Usage
noms = nommer_clusters(textes, labels)
for cluster_id, nom in noms.items():
print(f"Cluster {cluster_id}: {nom}")
indices = [i for i, l in enumerate(labels) if l == cluster_id]
for idx in indices:
print(f" - {textes[idx]}")Complete clustering pipeline
The three steps—embedding, clustering, naming—naturally come together in a single class. The methode parameter accepts "auto", which delegates the choice of group count to HDBSCAN, or a numeric value passed as a string when you want to impose a K on K-Means. The returned dictionary contains the labels, readable names, and two counters that immediately tell you whether the result is worth using: an n_outliers close to the total means your corpus lacks dense structure, and you should switch back to K-Means.
class PipelineClustering:
def __init__(self, model: str = "text-embedding-3-large"):
self.client = OpenAI()
self.model = model
def analyser(self, textes: list[str], methode: str = "auto") -> dict:
"""Complete pipeline: embedding → clustering → naming."""
# 1. Embeddings
response = self.client.embeddings.create(
input=textes, model=self.model
)
embs = np.array([
d.embedding for d in sorted(response.data, key=lambda x: x.index)
])
# 2. Clustering
if methode == "auto":
clusterer = hdbscan.HDBSCAN(min_cluster_size=3)
labels = clusterer.fit_predict(embs)
else:
km = KMeans(n_clusters=int(methode), random_state=42)
labels = km.fit_predict(embs)
# 3. Name clusters
noms = nommer_clusters(textes, labels)
return {
"labels": labels.tolist(),
"noms": noms,
"n_clusters": len(set(l for l in labels if l >= 0)),
"n_outliers": list(labels).count(-1)
}
pipeline = PipelineClustering()
resultats = pipeline.analyser(textes)Summary
- Use K-Means when you know the number of clusters
- Use HDBSCAN to automatically detect groups and outliers
- The silhouette score helps choose the right K
- An LLM automatically names discovered clusters
- Embeddings make clustering semantic, not just lexical