Sarvam AI's Indic DiarBench is here: How it helps developers
What's the story
Sarvam AI has launched Indic DiarBench, an open benchmark dataset for automatic speech recognition (ASR) and speaker diarization across 22 Indian languages. The dataset, which is available on Hugging Face, features around 108 hours of natural multi-speaker conversations. It addresses a gap in current speech datasets where ASR and speaker diarization are typically evaluated separately.
Overview
Dataset features conversations with 485 unique speakers
Indic DiarBench is centered on conversations with two to nine speakers, featuring rapid turn-taking, backchanneling, interruptions, and overlapping speech.
The dataset includes around 81 hours of meeting conversations with 485 unique speakers from 189 districts across India.
It also has about 28 hours of in-the-wild conversations covering the 10 most-spoken Indian languages with around 750 unique speakers.
Annotation and evaluation
It supports code-mixed speech
Each recording in the Indic DiarBench dataset underwent a five-stage annotation process.
This included creating bootstrap transcripts with multiple ASR models, human verification of words, timestamps, and speaker labels.
The dataset also supports code-mixed speech with transcriptions in native Indic scripts and Roman script for English words.
For evaluation, it uses three metrics: Diarization Error Rate (DER), concatenated minimum-permutation Word Error Rate (cpWER), and Word Diarization Error Rate (WDER).
Future plans
Indic DiarBench will be presented at Interspeech 2026
The Indic DiarBench dataset is significant as it allows for joint evaluation of ASR and speaker diarization tasks.
It will be presented at Interspeech 2026, with the research paper available on arXiv.
The benchmark aims to address a gap in existing speech datasets that reflects real-world conversational scenarios in India.