Voice Activity Detection (VAD)

benchflow-ai/skillsbenchMultimedia ProcessAudit passedSnapshot 7589a30b691a

Summary

Detect speech segments in audio using VAD tools like Silero VAD, SpeechBrain VAD, or WebRTC VAD. Use when preprocessing audio for speaker diarization, filtering silence, or segmenting audio into speech chunks. Choose Silero VAD for short segments, SpeechBrain VAD for general purpose, or WebRTC VAD for lightweight applications.

SKILL.md

Voice Activity Detection (VAD)

Overview

Voice Activity Detection identifies which parts of an audio signal contain speech versus silence or background noise. This is a critical first step in speaker diarization pipelines.

When to Use

Preprocessing audio before speaker diarization
Filtering out silence and noise
Segmenting audio into speech chunks
Improving diarization accuracy by focusing on speech regions

Available VAD Tools

1. Silero VAD (Recommended for Short Segments)

Best for: Short audio segments, real-time applications, better detection of brief speech

import torch

# Load Silero VAD model
model, utils = torch.hub.load(
    repo_or_dir='snakers4/silero-vad',
    model='silero_vad',
    force_reload=False,
    onnx=False
)
get_speech_timestamps = utils[0]

# Run VAD
speech_timestamps = get_speech_timestamps(
    waveform[0],  # mono audio waveform
    model,
    threshold=0.6,  # speech probability threshold
    min_speech_duration_ms=350,  # minimum speech segment length
    min_silence_duration_ms=400,  # minimum silence between segments
    sampling_rate=sample_rate
)

# Convert to boundaries format
boundaries = [[ts['start'] / sample_rate, ts['end'] / sample_rate]
              for ts in speech_timestamps]

Advantages:

Better at detecting short speech segments
Lower false alarm rate
Optimized for real-time processing

2. SpeechBrain VAD

Best for: General-purpose VAD, longer audio files

from speechbrain.inference.VAD import VAD

VAD_model = VAD.from_hparams(
    source="speechbrain/vad-crdnn-libriparty",
    savedir="/tmp/speechbrain_vad"
)

# Get speech segments
boundaries = VAD_model.get_speech_segments(audio_path)

import webrtcvad

vad = webrtcvad.Vad(2)  # Aggressiveness: 0-3 (higher = more aggressive)

# Process audio frames (must be 10ms, 20ms, or 30ms)
is_speech = vad.is_speech(frame_bytes, sample_rate)

def postprocess_boundaries(boundaries, min_dur=0.30, merge_gap=0.25):
    """
    boundaries: list of [start_sec, end_sec]
    min_dur: drop segments shorter than this (sec)
    merge_gap: merge segments if silence gap <= this (sec)
    """
    # Sort by start time
    boundaries = sorted(boundaries, key=lambda x: x[0])

    # Remove short segments
    boundaries = [(s, e) for s, e in boundaries if (e - s) >= min_dur]

    # Merge close segments
    merged = [list(boundaries[0])]
    for s, e in boundaries[1:]:
        prev_s, prev_e = merged[-1]
        if s - prev_e <= merge_gap:
            merged[-1][1] = max(prev_e, e)
        else:
            merged.append([s, e])

    return merged

Tool	Best For	Pros	Cons
Silero VAD	Short segments, real-time	Better short-segment detection	Requires PyTorch
SpeechBrain VAD	General purpose	Reliable, well-tested	May miss short segments
WebRTC VAD	Lightweight apps	Fast, lightweight	Less accurate, requires specific frame sizes

# After VAD, extract embeddings only for speech segments
for start, end in vad_boundaries:
    segment_audio = waveform[:, int(start*sr):int(end*sr)]
    embedding = speaker_model.encode_batch(segment_audio)
    # ... continue with clustering

Voice Activity Detection (VAD)

Summary

SKILL.md

Voice Activity Detection (VAD)

Overview

When to Use

Available VAD Tools

1. Silero VAD (Recommended for Short Segments)

2. SpeechBrain VAD

3. WebRTC VAD

Postprocessing VAD Boundaries

Choosing the Right VAD

Common Issues and Solutions

Integration with Speaker Diarization

Related skills