Nava

Persian speech-to-text. Nava is openai/whisper-large-v3-turbo (809M parameters) fine-tuned on 579 hours of filtered Persian speech.

• Strongest on conversational Persian among the four open models tested here.
• Writes standard Persian spelling, with half-spaces where they belong: «می‌شود», «سازمان‌ها».
• Runs wherever Whisper runs: transformers, faster-whisper, whisper.cpp.

This is a preview from a single training run. Read Limitations before relying on it.

Repo	Use it with
Nimruz/Nava (this one)	transformers
Nimruz/Nava-ct2	faster-whisper / CTranslate2
Nimruz/Nava-ggml	whisper.cpp

Results

Word error rate (%), with character error rate ignoring spaces in brackets. Lower is better.

Model	Params	C1Tech (conversational)	PSRB (mixed, noisy)	FLEURS fa (read)	Common Voice 22 fa (read)
Nava	809M	11.7 (3.8)	17.2 (5.8)	8.7 (2.3)	13.2 (3.3)
Reza2kn/Shenava-Koochik-v1.5	114M	13.4 (5.2)	22.5 (9.0)	8.1 (3.0)	12.5 (3.2)
mehdi-hf/nemotron-asr-streaming-farsi	600M	12.4 (4.5)	23.1 (9.3)	9.4 (3.2)	18.4 (5.4)
givia/whisper-large-v3-turbo-persian	809M	13.4 (5.1)	22.8 (9.9)	10.1 (5.2)	18.3 (6.4)
openai/whisper-large-v3-turbo (base)	809M	not run	not run	19.7 (6.2)	47.2 (26.8)

Test sets:

• C1Tech/Persian-ASR-Benchmark, all 2,101 clips.
• PartAI/PSRB, all 344 clips.
• FLEURS fa_ir test, all 871 clips.
• Common Voice 22 Persian test, a fixed random sample of 1,500 of the 10,676 clips.

Every model was run here, with the same scorer, on the same clips. See How the scores were computed for the method and for the same table under plain scoring.

Quick start

Input is 16 kHz mono audio. Always pass the language; Nava was trained with the Persian prompt only.

transformers

import soundfile as sf
import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor

model_id = "Nimruz/Nava"
processor = WhisperProcessor.from_pretrained(model_id)
model = WhisperForConditionalGeneration.from_pretrained(model_id).to("cuda").half()

wav, sr = sf.read("audio.wav", dtype="float32")  # 16 kHz mono, up to 30 seconds
features = processor(wav, sampling_rate=16000, return_tensors="pt").input_features
ids = model.generate(features.to("cuda").half(), language="fa", task="transcribe", max_new_tokens=440)
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

On a machine without a GPU, drop .to("cuda") and .half().

Any length, any audio format

The script in this repo decodes the file, cuts long audio at quiet points, and joins the text:

git clone https://huggingface.co/Nimruz/Nava && cd Nava
pip install torch transformers
python transcribe.py interview.mp3
python transcribe.py interview.mp3 --digits fa     # numbers, times and dates in digits

It uses ffmpeg to read audio if ffmpeg is installed, and librosa otherwise.

faster-whisper

from faster_whisper import WhisperModel

model = WhisperModel("Nimruz/Nava-ct2", device="cuda", compute_type="float16")
segments, _ = model.transcribe(
    "audio.wav", language="fa", beam_size=1,
    without_timestamps=True, condition_on_previous_text=False,
)
print(" ".join(s.text.strip() for s in segments))

On CPU use device="cpu", compute_type="float32", or "int8" for less memory.

whisper.cpp

huggingface-cli download Nimruz/Nava-ggml ggml-nava-q5_0.bin --local-dir .
whisper-cli -m ggml-nava-q5_0.bin -l fa -nt -f audio.wav

File	Size	Notes
ggml-nava.bin	1.51 GB	16-bit float
ggml-nava-q8_0.bin	0.81 GB	8-bit quantized
ggml-nava-q5_0.bin	0.53 GB	5-bit quantized

whisper.cpp expects 16 kHz 16-bit WAV: ffmpeg -i in.mp3 -ar 16000 -ac 1 -c:a pcm_s16le audio.wav.

Which one to pick

You have	Use
An NVIDIA GPU and Python	transformers or faster-whisper
A CPU server	faster-whisper
A laptop or phone, no Python	whisper.cpp with the q5_0 or q8_0 file

Each export was checked against the original model on 10 FLEURS test clips: the transformers weights gave identical text on all 10, and the CTranslate2 and three whisper.cpp files on 9 of 10. The whisper.cpp files were built and tested with whisper.cpp v1.7.4.

Things to know when running it

Audio longer than 30 seconds

Whisper's built-in long-form mode relies on timestamp tokens, and Nava was fine-tuned without them. Do not use that mode: on the 13 FLEURS clips over 30 seconds it scored 31.9% word error that way.

Cut long audio into pieces of at most 30 seconds at quiet points and transcribe each piece. transcribe.py does this, and with it the same 13 clips score 11.1%. A voice-activity splitter such as vad_filter=True in faster-whisper should serve the same purpose, but that path has not been tested with Nava.

Numbers, dates and times

Nava writes numbers the way they are spoken: «هزار و چهارصد و دو». The formatter in this repo turns them into digits after transcription:

from nimruz.itn import itn

itn("سال هزار و چهارصد و دو بیست و پنج درصد رشد داشتیم")   # سال ۱۴۰۲ ۲۵ درصد رشد داشتیم
itn("ساعت ده و نیم دوازده مرداد میام")                      # ساعت ۱۰:۳۰ ۱۲ مرداد میام
itn("قیمتش دو میلیون و پونصد هزار تومنه", digits="ascii")    # قیمتش 2500000 تومنه

It is rule-based and conservative: lone small words such as «یک» and «نه» stay as words, and ordinals such as «بیست و پنجم» are left alone.

Beam search

Greedy decoding is the default everywhere above. Five beams lower word error by 0.1 to 0.3 points (11.6 / 16.9 / 8.6 / 13.1 on the four test sets) at about half the speed.

How the scores were computed

Persian sources disagree on word spacing: «می‌کنم», «می کنم» and «میکنم» are the same word. Scoring those as errors measures the spelling convention, not the recognition. So before comparing, reference and hypothesis both go through the same steps:

1. Rewrite spacing variants to the standard half-space forms.
2. Fold Arabic and Persian letter variants, and remove diacritics and punctuation.
3. Spell out digits as words.
4. Treat the half-space as a space.

The same table with step 1 switched off (word error rate, %):

Model	C1Tech	PSRB	FLEURS fa	Common Voice 22 fa
Nava	17.0	17.2	9.7	20.6
Reza2kn/Shenava-Koochik-v1.5	18.0	22.7	8.9	19.8
mehdi-hf/nemotron-asr-streaming-farsi	17.3	23.3	10.3	25.7
givia/whisper-large-v3-turbo-persian	16.4	26.1	10.3	23.4

Notes on the comparison:

• Numbers for the other models were measured here and can differ from their own cards, which use other normalizers and clip selections.
• Decoding was greedy for every Whisper model, with long clips cut at quiet points for Nava and Whisper's long-form mode for the others. Shenava used its CTC head; Nemotron used its 1.12 s look-ahead setting.
• Shenava was used to filter Nava's training data (see below), so the two are not fully independent. It was not used on any test set.

To score your own outputs the same way, write one JSON object per line with ref and hyp fields to results/<model-name>/<test-set>.jsonl and run:

pip install jiwer
python score.py

Training data

Source	Kind	Raw hours	Used hours
PerSets/youtube-persian-asr	YouTube subtitles, conversational	385	294 (824 videos)
PerSets/filimo-persian-asr	Film and series subtitles	244	176 (466 titles)
MahtaFetrat/Mana-TTS, parts 1 to 69	Magazine readings, one speaker	103	60
Common Voice 25 fa, validated	Read sentences	372	49 (1,120 speakers)
Total		1,104	579

How clips were chosen:

• Agreement check. Every clip was transcribed by an independent model (Shenava Koochik v1.5). Clips whose label differed from that transcript by more than 30% of characters were dropped. This removed about 120 hours, mostly subtitles that do not line up with the audio.
• Text checks. Clips with digits, Latin letters, URLs, stage directions, speaker labels or runs of repeated characters were dropped.
• Speech-rate and audio checks. Clips under 0.8 seconds, implausibly slow or fast for their text, or near silent were dropped.
• Diversity caps. At most 40 minutes per Common Voice speaker and 3 recordings of the same sentence; Mana-TTS capped at 60 hours so one voice does not dominate.
• Test protection. Common Voice test speakers and test sentences were excluded. Mana-TTS parts 70 to 77 (a public leaderboard's test set) were not used. 2% of YouTube and Filimo recordings were held out for development.

Training text was normalized to Persian letter forms, no diacritics, no punctuation, and one spacing convention (the standard half-space forms). The rules and word list for that are in nimruz/ortho.py and assets/lexicon.json.

Training

Base model	openai/whisper-large-v3-turbo
Trainable	Decoder and encoder layers 9 to 32 (conv stem and first 8 encoder layers frozen)
Steps	10,386 optimizer steps, effective batch 32 (8 × 4 accumulation)
Audio seen	About 1,590 hours, roughly 2.75 passes over the data
Learning rate	1e-5 peak, 400 warmup steps, cosine decay to 1e-6
Optimizer	AdamW, betas (0.9, 0.98), weight decay 0.01, gradient clipping 1.0
Precision	bfloat16 autocast
Augmentation	SpecAugment (time and feature masking, 5% each)
Sample packing	Neighbouring clips concatenated up to 30 s; 15% of samples left as single clips
Compute	6.5 GPU-hours on one GPU (A100 for the first 500 steps, L40S after)

Development word error went from 18.3% to 8.3% on FLEURS dev and from 45.5% to 15.1% on held-out conversational clips (250 clips each). The conversational score was 14.1% at step 9,000 and 15.1% at the end; the released weights are the final step, which scored better on the FLEURS test set.

Limitations

• No punctuation. Output is a plain stream of words.
• Persian only. English words are written in Persian script or mis-transcribed.
• No timestamps, and no built-in long-form decoding (see above).
• Numbers are spelled out unless the formatter is applied.
• Spacing is not fully uniform. Some forms are still written joined, such as «شدهاند» for «شده‌اند».
• Behaviour on silence, music and non-speech audio has not been tested. Whisper models can produce text where there is no speech.
• One training run. No second seed and no hyperparameter search.
• Evaluation is on four public sets only, and the Common Voice figure is from a 1,500-clip sample.

What's in this repo

Path	What it is
model.safetensors, config.json, tokenizer files	The model, half precision
transcribe.py	Transcribe audio of any length and format
nimruz/itn.py	Spoken numbers, dates and times to digits
nimruz/textnorm.py, nimruz/ortho.py, assets/lexicon.json	Text normalization and spacing rules used for training text and scoring
nimruz/scoring.py, score.py	The scorer behind the tables above
assets/*.png	The charts on this page

Licence and data

Nava is released under the MIT licence, the same as its base model. The four training datasets are published under CC0. The YouTube and Filimo sets consist of audio from online video with its subtitles; check that this suits your use before deploying commercially.

Acknowledgements

Thanks to the authors of the training datasets, to the Shenava project for the model used in filtering, and to the authors of the FLEURS, Common Voice, C1Tech and PSRB test sets.