CIIL/Anuvani-Diarization

---

Who spoke when, for multi-speaker audio in the 22 scheduled languages of India and English.

Anuvani-Diarization finds who spoke when in recordings of conversations, calls and meetings: it splits the audio into speaker turns and labels each turn with an anonymous speaker, in a standard RTTM file. It is built for the 22 scheduled languages of India and English. On Indic DiarBench, the public benchmark for those languages, it has the lowest error of every system we ran, evaluated zero-shot: nothing in it was trained, tuned or selected on the benchmark.

Release 1.0.0 comes with:

• anuvani_diar, a pip package that runs the model in a few lines of Python on a Rust engine, on every GPU you give it (or the CPU).
• triton_inference/, a serving kit for many users: Triton Inference Server and a FastAPI service, on any number of NVIDIA GPUs on one machine or several.
• The original Python program (reference/), whose outputs the engine reproduces.

Key capabilities:

• The lowest error of every system we ran on Indic DiarBench: 16.78 DER (duration-weighted) and 17.11 (pooled) on all 1,164 recordings, zero-shot, against 17.18 to 39.41 for the nine other systems we ran; the most accurate measured system in 14 of the 22 languages.
• Any length, any number of speakers: from seconds to many hours; up to 4 people talking at once within each 16-second window, and no limit on the total number of speakers.
• Runs on what you have: any NVIDIA GPU from the Volta generation to Blackwell, as many as you have, on one machine or several; the CPU works too, more slowly. By default it uses every GPU, and you can choose which.
• Fast: decoding, windowing, batching across GPUs and clustering run in Rust; only the two neural networks run in PyTorch.
• Reproducible: on the setup the release's outputs were produced with, the output is byte-identical to the original program; on any other GPU or PyTorch version every GPU checks its networks at start-up against the release's own outputs.
• Open source code under MIT OR Apache-2.0; the weights are for non-commercial use (CC BY-NC 4.0).

Model overview

Property	Value
Task	Speaker diarization: who spoke when; one RTTM file per recording
Languages	Built and evaluated for the 22 scheduled languages of India and English
Segmentation network	DiariZen's pruned WavLM-Large plus Conformer end-to-end network, fine-tuned (69.5M parameters)
Voiceprint network	WeSpeaker ResNet34 (6.6M parameters)
Clustering	VBx clustering with the base model's PLDA and default settings (AHC threshold 0.6, Fa 0.07, Fb 0.8); nothing after the network is tuned
Total parameters	76.1M
Analysis window	16 seconds, one every 1.6 seconds; up to 4 overlapping speakers per window
Input	any audio file libsndfile reads (WAV, FLAC, OGG, MP3), any length; converted to 16 kHz mono
Output	RTTM (speaker label, start, duration); in Python also a list of (start, end, speaker)
Hardware	any NVIDIA GPU from Volta to Blackwell with about 12 GB of memory free, any number of them, on one machine or several; or the CPU
Licence	code MIT OR Apache-2.0; weights CC BY-NC 4.0 (segmentation) and CC BY 4.0 (PLDA, voiceprints)
Paper	arXiv 2610.20327 · Hugging Face paper page

Benchmark results

All numbers are diarization error rates (DER, %): lower is better. Zero collar, overlap scored, NIST md-eval semantics (identical to dscore e02f949).

Indic DiarBench

All 1,164 recordings (107.7 hours, 22 Indian languages), dataset revision 92877bad.

#	System	Maker	DER, duration-weighted	DER, pooled	Source
1	Sarvam AI	Sarvam AI	16.0		paper
2	Anuvani-Diarization 1.0	CIIL	16.78	17.11	this model, zero-shot
3	Precision-2	pyannoteAI	17.18	17.58	run by us
4	DiariZen WavLM-Large s80 md v2 (the base model)	BUT Speech@FIT	17.99	18.31	run by us
5	Nemotron 3 Diarization	NVIDIA	18.38	18.71	run by us
6	speaker-diarization-community-1	pyannote	21.22	21.65	run by us
7	Streaming Sortformer 4spk v2.1	NVIDIA	22.14	22.34	run by us
8	DiCoW v3.3	BUT Speech@FIT	22.62	22.76	run by us
9	AWS Transcribe	Amazon	23.5		paper
10	3D-Speaker CAM++ diarization	Alibaba Tongyi	23.67	24.39	run by us; 2 recordings failed
11	Deepgram Nova-3	Deepgram	32.0		paper
12	MOSS-Transcribe-Diarize 0.9B	OpenMOSS	33.76	34.52	run by us; 14 recordings failed
13	Azure STT	Microsoft	34.8		paper
14	ElevenLabs Scribe	ElevenLabs	35.0		paper
15	GPT-4o Transcribe	OpenAI	36.2		paper
16	Sortformer 4spk v1	NVIDIA	39.41	39.66	run by us; 40 recordings failed
17	AssemblyAI Universal-2	AssemblyAI	40.5		paper
18	Gemini 3 Pro	Google	74.0		paper

• Run by us: every system on the same 1,164 recordings with one scorer: 0 s collar, overlap scored, no speaker-count hint; recordings a system failed on are scored as empty output.
• Paper: Indic DiarBench (Sarvam AI and AI4Bharat, arXiv 2607.23808), Table 3, duration-weighted, run as joint transcription and diarization services. The paper does not name Sarvam AI's model.
• This model: evaluated zero-shot. Nothing in it was trained, tuned or selected on Indic DiarBench: the segmentation network was adapted on other recordings, its recipe and checkpoint were fixed on non-benchmark recordings, and clustering uses the base model's default settings. The system was frozen before it was first scored on the benchmark. Earlier versions of this system were also scored on Indic DiarBench.
• Pooled 95% confidence interval: 15.80 to 18.45 (bootstrap over the 590 source recordings). The paired differences against the three strongest systems we ran all exclude zero: -0.47 points against Precision-2, -1.21 against DiariZen md-v2 and -1.60 against Nemotron 3 Diarization.
• Scorer choice changes the number. pyannote.metrics reads about 0.20 higher on Indic DiarBench, because it does not merge the benchmark's overlapping same-speaker reference segments (13.7 minutes in 196 recordings).

Public benchmarks

This model against its base model on six international test sets, both run by us under the same conditions with the base model's clustering.

Test set	Files	DiariZen WavLM-Large s80 md v2 (the base model)	Anuvani-Diarization 1.0
VoxConverse v0.3 test	232	9.17	9.32
NOTSOFAR-1 eval (one single-channel device per meeting)	129	18.26	17.41
MagicData-RAMC test	43	10.97	11.13
AMI test, single distant mic	16	13.93	14.00
AISHELL-4 test	20	10.06	9.85
AliMeeting test, far-field	20	10.74	10.44
Mean		12.19	12.03

The Indic adaptation costs no accuracy on these sets on average: better on three, worse on three.

This model on every benchmark

Benchmark	DER
Indic DiarBench, all 1,164 recordings, duration-weighted	16.78
Indic DiarBench, all 1,164 recordings, pooled	17.11
Indic DiarBench, near-field meetings, pooled	19.05
Indic DiarBench, far-field meetings, pooled	18.64
Indic DiarBench, in-the-wild video, pooled	11.51
VoxConverse v0.3 test	9.32
AISHELL-4 test	9.85
AliMeeting test, far-field	10.44
MagicData-RAMC test	11.13
AMI test, single distant mic	14.00
NOTSOFAR-1 eval (one single-channel device per meeting)	17.41
Average of the six public sets	12.03

Getting started

opens the quick-start notebook (notebook.ipynb) in Google Colab: choose a GPU runtime and run every cell.

On your own machine you need Linux x86_64 with glibc 2.28 or newer, Python 3.10 or newer and PyTorch 2.1 or newer; an NVIDIA GPU is strongly recommended (the CPU works, much more slowly).

pip install -U huggingface_hub
hf download CIIL/Anuvani-Diarization wheels/anuvani_diar-1.0.0-cp310-abi3-manylinux_2_28_x86_64.whl --local-dir .
pip install wheels/anuvani_diar-1.0.0-cp310-abi3-manylinux_2_28_x86_64.whl

from anuvani_diar import Diarizer

# every visible GPU, or the CPU when there is none
diarize = Diarizer()
# any length; WAV, FLAC, OGG, MP3
result = diarize("meeting.wav")
result.save_rttm("meeting.rttm")
# [(start, end, speaker), ...]
print(result.speakers, result.segments[:3])

# many files; results arrive as each one finishes
for r in diarize.run(["a.wav", "b.flac"], out_dir="rttm/"):
    print(r.uri, r.ok)

Choosing devices: Diarizer(devices=2) uses the first two GPUs, Diarizer(devices=[0, 3]) GPUs 0 and 3, Diarizer(devices="cpu") the CPU. The engine spreads every file's windows over all of them.

• Command line: anuvani-diar --out rttm/ meeting.wav, or --in-dir recordings/; --devices 2, --devices 0,3 or --devices cpu choose the devices.
• What it downloads: the first Diarizer() fetches model/ (307 MB) once from this repository, at the release's tag, and checks every file's sha256 against the package's own list. No account is needed. It also fetches the small config.json, which the Hub counts as a download of the model; nothing else is sent.
• Install check: anuvani-diar-check diarizes the 12 check recordings and compares the results with the release's fingerprints.
• PyTorch: installing the wheel into an environment without PyTorch pulls PyPI's default build; for PyTorch 2.14.1 that is a CUDA 13.0 build, which needs a recent NVIDIA driver. With an older driver, install a CUDA 12 build first, e.g. pip install torch --index-url https://download.pytorch.org/whl/cu126. Google Colab already has PyTorch.

Serving at scale

triton_inference/ deploys the model for many users with Docker Compose:

• one Triton Inference Server per GPU holds the two networks;
• a FastAPI service holds the Rust engine, and it is the only thing clients talk to (uploads, job ids, RTTMs, a token, limits, metrics);
• it uses every GPU of the machine by default (./diar up 2 or ./diar up --gpus 0,3 to choose), and the GPUs of other machines too (./diar up --tritons-only on those, ./diar up --remote ... on the service's machine).

cd triton_inference && chmod +x diar
# the Triton image this machine's GPUs need, the service, every GPU
./diar doctor && ./diar build && ./diar up
# the 12 check recordings through the service
./diar gate
curl -s -H "Authorization: Bearer $(./diar token)" --data-binary @meeting.wav "localhost:8180/v1/diarize?name=meeting.wav"

The kit's default lane runs part of the networks in FP16 and TF32. It is not byte-identical to the release, and its output can differ slightly from run to run. ./diar up --exact runs the exact lane. Details: triton_inference/README.md.

Best practices

1. Devices

Leave devices on "auto" to use every GPU, or pass a number or a list to share a machine. One process per machine is best: the engine already keeps every GPU busy and spreads each file over all of them.

2. Many files

Give run() all the files at once rather than calling the model once per file: the engine decodes, schedules and clusters them in parallel, and results arrive as each file finishes.

3. Long recordings

Any length works, from seconds to many hours. A single long file finishes faster with more GPUs, because its windows spread over all of them; decoding and the clustering run on the CPU before and after the networks.

4. Speed and exactness

The pip package runs the exact lane (the release's FP32 networks). The serving kit defaults to the faster ffn16 lane, which serves about 2.1 times as much audio per second as the exact lane on the same GPUs; --exact keeps the exact lane.

5. Audio formats

WAV, FLAC, OGG and MP3 are read directly; convert M4A, AAC and MP4 audio to WAV or FLAC first.

6. Checking an installation

anuvani-diar-check (and ./diar gate in the serving kit) diarize the 12 check recordings. Identical results confirm the setup the release's outputs were produced with; anywhere else the networks pass a near-identical check and the results may differ slightly.

Model data

Training data

• Segmentation network: DiariZen's WavLM-Large and Conformer network (BUT-FIT/diarizen-wavlm-large-s80-md-v2), fine-tuned on Indian telephone conversations (2-person calls in 8 languages: as, bn, gu, hi, ml, mr, pa, ta) and international meeting corpora.
• Voiceprint network: pyannote/wespeaker-voxceleb-resnet34-LM, unchanged.
• PLDA model and clustering settings: BUT Speech@FIT's, unchanged.
• Indic DiarBench: not used for training, tuning or selection; audio-fingerprint screening found no audio shared between the training data and the benchmark.

Evaluation data

Indic DiarBench (Sarvam AI and AI4Bharat): 1,164 recordings, 107.7 hours, in 22 Indian languages and three recording conditions, at the pinned dataset revision 92877bad. The public sets are listed in the tables above.

Exactness and determinism

• FP32, TF32 off, deterministic kernels, seed 3407 reset per file.
• Byte-identical to the research pipeline the published numbers come from.
• The package's default (exact) lane and the kit's exact lane give RTTMs byte-identical to the original program on the setup the release's outputs were produced with (PyTorch 2.1.1 with CUDA 12.1): all 1,945 benchmark files from cached outputs, and 1,973 audio inputs, including 1-, 3- and 6-hour files and damaged files, both locally and through Triton.
• Other setups (other GPUs, other PyTorch versions) run the same FP32 networks with the same settings, but cannot be byte-identical. At start-up, each GPU's outputs are checked against the release's own and must agree within fixed bounds; the package then warns that its output is near-identical, and Diarizer(strict=True) refuses instead. On other GPU models, rare near-ties in clustering could flip a file; the install checks show whether a setup reproduces the release byte for byte.

Usage and limitations

Intended usage

• Conversation and meeting analysis: speaker turns for calls, interviews, panels, classrooms and meetings in Indian languages and English.
• Transcription pipelines: attributing an ASR transcript's words to speakers, by aligning them with the RTTM.
• Media and archives: indexing radio, television and oral-history recordings by speaker turn.
• Research: a strong, reproducible baseline for speaker diarization in Indian languages.

Limitations

• Overlap cap: at most 4 people talking at the same moment within a 16-second window. There is no limit on the total number of speakers.
• Indian fine-tuning data is narrow: 2-person telephone calls in 8 languages (as, bn, gu, hi, ml, mr, pa, ta). There are no Indian multi-person meetings, and 14 of Indic DiarBench's 22 languages are unseen in fine-tuning.
• Languages and conditions: it trails the best other system we ran most in Maithili (+1.98 points), Sindhi (+1.80) and Bodo (+1.56), and on in-the-wild video, where Precision-2 leads (10.78 against 11.51 pooled).
• Annotation convention: Indic DiarBench's references mark whole utterances, while this model, like other diarizers, splits speech at pauses; part of its error on the benchmark comes from that difference.
• Speaker identity inside overlaps and very short turns is the largest remaining error source.
• Single training seed.
• Evaluation history: earlier versions of this system were also scored on Indic DiarBench.
• Formats: M4A/AAC/MP4 audio is not decoded; convert it to WAV or FLAC first.

Ethical considerations and risks

• Privacy and consent: diarization tells apart the voices in a recording. Use it on recordings whose speakers have agreed to it, and follow the data protection law that applies to you.
• No identities: the speaker labels are anonymous clusters within one recording; the model does not identify people and does not link speakers across recordings. Using its output to identify or track people needs separate consent and a separate, appropriately evaluated system.
• Uneven accuracy: error rates vary with language, recording condition and the number of people talking; check the results on your own audio before relying on them.
• Licence: the weights are for non-commercial use only (CC BY-NC 4.0, inherited from DiariZen).

Benefits

At release, the most accurate openly available speaker diarization for Indian languages on Indic DiarBench, with reproducible results, open source code and a deployment that scales from one GPU to many machines.

Repository layout

Path	What
model/	the weights (safetensors) and MANIFEST.json (every file's sha256)
model.safetensors.index.json	the index of the tensors in model/'s two safetensors files (the Hub reads the parameter count from it)
config.json	the release's description (version, wheel, window, parameters, licences); the package fetches it with model/
wheels/	the pip package (anuvani_diar-1.0.0-cp310-abi3-manylinux_2_28_x86_64.whl)
notebook.ipynb, notebooks/quickstart.ipynb	install and diarize in a notebook (the same notebook; the first opens in Colab from this page)
triton_inference/	the serving kit: the diar command, the FastAPI service, the Triton images, Docker Compose, checks, a load test
golden/audio/	the 12 Indic DiarBench recordings of the install checks
reference/	the original Python program (diarize.py), its pickled weights and its install check
src/	the source of the pip package and the serving kit (Rust and Python), its tests and developer documents
CHANGELOG.md	release notes
LICENSE-MIT, LICENSE-APACHE	the code's licence (MIT OR Apache-2.0)
THIRD_PARTY_NOTICES.md, LICENSES/	third-party code, bundled libraries and Rust crates, with their licence texts
third_party_sources/	the source of the LGPL libraries the wheel bundles, and the script that built libsndfile

Licence and attribution

• Segmentation network: a fine-tuned derivative of BUT-FIT/diarizen-wavlm-large-s80-md-v2, under CC BY-NC 4.0 (non-commercial). DiariZen's weights carry that licence because some of their training data is research-only.
• PLDA model: BUT Speech@FIT, CC BY 4.0.
• Speaker-embedding model: pyannote/wespeaker-voxceleb-resnet34-LM, CC BY 4.0.
• Indic DiarBench audio (golden/audio): derived from Sarvam AI's Indic DiarBench (CC BY 4.0). Cite Mehendale et al., Indic DiarBench, Interspeech 2026 (arXiv 2607.23808).
• DiariZen: cite Han et al., ICASSP 2025, and arXiv 2505.24111 / 2506.18623.
• Code: MIT OR Apache-2.0, at your option (LICENSE-MIT, LICENSE-APACHE).
  • It contains or ports code from DiariZen, torchaudio, pyannote.audio, WeSpeaker, VBx, SciPy, NumPy and python-soundfile.
  • The wheel bundles numpy 1.26.4's OpenBLAS (BSD-3) with libgfortran (GPL-3 with the GCC Runtime Library Exception) and libquadmath (LGPL-2.1+), and libsndfile 1.2.2 (LGPL-2.1) with its codecs.
  • Their licences and notices are in THIRD_PARTY_NOTICES.md and LICENSES/, and in the wheel.
  • The source of the LGPL ones (libsndfile with its codecs, libquadmath) is in third_party_sources/.

Acknowledgement

Anuvani-Diarization was conceived and developed by Stephen Fernandes under the leadership and guidance of Dr. Narayan Kumar Choudhary. We thank Dr. Choudhary and the Central Institute of Indian Languages (CIIL) for their assistance, for the compute they provided and for their continued support.

Citation

If you use this model, please cite the paper:

@misc{fernandes2026anuvanidiarization,
  title={Anuvani-Diarization: Zero Shot Speaker Diarization and Annotation Conventions Across 22 Languages},
  author={Stephen Fernandes and Narayan Choudhary},
  year={2026},
  eprint={2610.20327},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2610.20327}
}