Choose and implement clustering algorithms for grouping speaker embeddings after VAD and embedding extraction. Compare Hierarchical clustering (auto-tunes speaker count), KMeans (fast, requires known count), and Agglomerative clustering (fixed clusters). Use Hierarchical clustering when speaker count is unknown, KMeans when count is known, and always normalize embeddings before clustering.
SKILL.md
Speaker Clustering Methods
Overview
After extracting speaker embeddings from audio segments, you need to cluster them to identify unique speakers. Different clustering methods have different strengths.
When to Use
After extracting speaker embeddings from VAD segments
Need to group similar speakers together
Determining number of speakers automatically or manually
Available Clustering Methods
1. Hierarchical Clustering (Recommended for Auto-tuning)
Best for: Automatically determining number of speakers, flexible threshold tuning
from scipy.cluster.hierarchy import linkage, fcluster
from scipy.spatial.distance import pdist
import numpy as np
# Prepare embeddings
embeddings_array = np.array(embeddings_list)
n_segments = len(embeddings_array)
# Compute distance matrix
distances = pdist(embeddings_array, metric='cosine')
# Create linkage matrix
linkage_matrix = linkage(distances, method='average')
# Auto-tune threshold to get reasonable speaker count
min_speakers = 2
max_speakers = max(2, min(10, n_segments // 2))
threshold = 0.7
labels = fcluster(linkage_matrix, t=threshold, criterion='distance')
n_speakers = len(set(labels))
# Adjust threshold if needed
if n_speakers > max_speakers:
for t in [0.8, 0.9, 1.0, 1.1, 1.2]:
labels = fcluster(linkage_matrix, t=t, criterion='distance')
n_speakers = len(set(labels))
if n_speakers <= max_speakers:
threshold = t
break
elif n_speakers < min_speakers:
for t in [0.6, 0.5, 0.4]:
labels = fcluster(linkage_matrix, t=t, criterion='distance')
n_speakers = len(set(labels))
if n_speakers >= min_speakers:
threshold = t
break
print(f"Selected: t={threshold}, {n_speakers} speakers")
Advantages:
Installs
0
Automatically determines speaker count
Flexible threshold tuning
Good for unknown number of speakers
Can visualize dendrogram
2. KMeans Clustering
Best for: Known number of speakers, fast clustering
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
import numpy as np
# Normalize embeddings
embeddings_array = np.array(embeddings_list)
norms = np.linalg.norm(embeddings_array, axis=1, keepdims=True)
embeddings_normalized = embeddings_array / np.clip(norms, 1e-9, None)
# Try different k values and choose best
best_k = 2
best_score = -1
best_labels = None
for k in range(2, min(7, len(embeddings_normalized))):
kmeans = KMeans(n_clusters=k, random_state=0, n_init=10)
labels = kmeans.fit_predict(embeddings_normalized)
if len(set(labels)) < 2:
continue
score = silhouette_score(embeddings_normalized, labels, metric='cosine')
if score > best_score:
best_score = score
best_k = k
best_labels = labels
print(f"Best k={best_k}, silhouette score={best_score:.3f}")
Advantages:
Fast and efficient
Works well with known speaker count
Simple to implement
Disadvantages:
Requires specifying number of clusters
May get stuck in local minima
3. Agglomerative Clustering
Best for: Similar to hierarchical but with fixed number of clusters
from sklearn.cluster import AgglomerativeClustering
from sklearn.metrics import silhouette_score
import numpy as np
# Normalize embeddings
embeddings_array = np.array(embeddings_list)
norms = np.linalg.norm(embeddings_array, axis=1, keepdims=True)
embeddings_normalized = embeddings_array / np.clip(norms, 1e-9, None)
# Try different numbers of clusters
best_n = 2
best_score = -1
best_labels = None
for n_clusters in range(2, min(6, len(embeddings_normalized))):
clustering = AgglomerativeClustering(n_clusters=n_clusters)
labels = clustering.fit_predict(embeddings_normalized)
if len(set(labels)) < 2:
continue
score = silhouette_score(embeddings_normalized, labels, metric='cosine')
if score > best_score:
best_score = score
best_n = n_clusters
best_labels = labels
print(f"Best n_clusters={best_n}, silhouette score={best_score:.3f}")
