Nvidia releases open Nemotron 3.5 Transcribe model: speech transcription in 40 languages, translation and speaker labeling in one model
Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .
Speech recognition with translation and speaker labels is packed into one open model that can be fine-tuned for a specific field.
According to Nvidia's description, the model transcribes speech in 40 languages, translates 31 of them to and from English, and tells speakers apart in 13; it handles recordings up to an hour and up to eight voices, and the form of the output is set by a text prompt. Weights are available on Hugging Face and ModelScope. According to Nvidia, the model ranks first among open ones on the Hugging Face Open ASR Leaderboard for English and showed the lowest speaker-attribution error on a 21-language ranking by David AI. Alongside it, the Granary v2 dataset and fine-tuning recipes are open: in an example with medical recordings, fine-tuning cut the word error rate by 45.8% relative to the best prompt. Nvidia promises quantized versions for RTX Spark, DGX Spark and Jetson AGX Thor later.
Sources 1