Close Menu
  • AI
  • Content Creation
  • Tech
  • Robotics
AI-trends.todayAI-trends.today
  • AI
  • Content Creation
  • Tech
  • Robotics
Trending
  • Potential Cognition AI Spherical Would Elevate $2B-Plus at $75B Valuation – Unite.AI
  • OpenRouter Cuts Three Suppliers After AI Immediate Publicity Findings – Unite.AI
  • A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels
  • Collectively AI Discloses Immediate Leak After Coding Agent Uncovered Credential – Unite.AI
  • Dragos Finds Immediate Leaks and Stolen Keys in AI Provide Chains
  • Lean and Mathlib Communities Certify Claude-Assisted Riemann Proof – Unite.AI
  • AWS Routes HyperPod Inference Throughout Areas From One Endpoint – Unite.AI
  • DailyAI Briefing: A Lean-checked Riemann proof, plus 4 extra
AI-trends.todayAI-trends.today
Home»Tech»NVIDIA Releases Nemotron 3.5 ASR: A 600M-Parameter Cache-Conscious Streaming Mannequin Transcribing 40 Language-Locales in Actual Time

NVIDIA Releases Nemotron 3.5 ASR: A 600M-Parameter Cache-Conscious Streaming Mannequin Transcribing 40 Language-Locales in Actual Time

Tech By Gavin Wallace06/06/20268 Mins Read
Facebook Twitter LinkedIn Email
NVIDIA Releases Llama Nemotron Nano 4B: An Efficient Open Reasoning
NVIDIA Releases Llama Nemotron Nano 4B: An Efficient Open Reasoning
Share
Facebook Twitter LinkedIn Email

NVIDIA’s Nemotron Speech staff has launched Nemotron 3.5 ASR. It’s a 600M-parameter streaming Computerized Speech Recognition (ASR) mannequin. A single checkpoint transcribes 40 language-locales in actual time. Punctuation and capitalization are in-built natively. The mannequin ships as open weights on Hugging Face. The license is OpenMDW-1.1. The structure is a Cache-Conscious FastConformer-RNNT.

What’s Nemotron 3.5 ASR

Nemotron 3.5 ASR extends nvidia/nemotron-speech-streaming-en-0.6b to many languages. It provides prompt-based language-ID conditioning to the bottom mannequin. That lets one 600M-parameter checkpoint cowl 40 language-locales. No per-language mannequin or model-swapping is required.

The mannequin targets two workloads. The primary is low-latency streaming for reside audio. The second is high-throughput batch transcription. Output is production-ready textual content with correct casing and punctuation. No separate punctuation-restoration step is required.

Picture supply: https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b

How Cache-Conscious FastConformer-RNNT Works

The mannequin has two foremost items. The primary is a Cache-Conscious FastConformer encoder with 24 layers. FastConformer is an environment friendly evolution of the Conformer structure. It makes use of linearly scalable consideration. The second piece is an RNNT (Recurrent Neural Community Transducer) decoder. RNNT emits textual content body by body as audio streams in.

The “cache-aware” design is the effectivity lever. Buffered streaming re-processes overlapping audio home windows at each step. That repeats the identical work and provides delay. This mannequin caches encoder self-attention and convolution activations as a substitute. It reuses these cached states as new audio arrives. So every audio body is processed precisely as soon as, with no overlap. Compute and end-to-end latency each drop, with out an accuracy penalty.

The Latency Knob: att_context_size

One inference setting controls the latency-accuracy tradeoff. It’s the consideration context dimension, att_context_size. Smaller context emits textual content sooner however sees much less future audio. Bigger context raises accuracy at increased latency.

The identical checkpoint covers the complete vary. Settings map to chunk sizes of 80ms, 160ms, 320ms, 560ms, and 1.12s. For instance, [56,0] provides an 80ms ultra-low-latency mode. The [56,13] setting provides 1.12s for highest accuracy. Groups decide the working level at inference time, with no retraining.

Language Detection and Protection

The 40 language-locales embody English, Spanish, German, and French variants. In addition they cowl Arabic, Japanese, Korean, Mandarin, Hindi, and Thai. A number of different European and Nordic languages are included too.

Language conditioning works two methods. Setting target_lang to a recognized locale often provides the most effective accuracy. Setting target_lang=auto lets the mannequin detect the language itself. In auto mode, it emits a language tag after terminal punctuation. One deployment can then transcribe mixed-language site visitors. No separate language-ID part is required.

Comparability

Product Firm Entry Native streaming Language protection Reported latency Pricing mannequin
Nemotron 3.5 ASR NVIDIA Open weights (OpenMDW-1.1), self-host; hosted on DeepInfra Sure — cache-aware FastConformer-RNNT 40 language-locales 80ms–1.12s, configurable at inference Free to self-host; usage-based by way of host
Whisper large-v3 OpenAI Open weights (MIT), self-host; API No — offline/batch ~99 languages Not streaming-native Self-host free; API ~$0.006/min (batch)
Nova-3 Deepgram Closed API; on-premise/self-host (enterprise) Sure — streaming + batch Multilingual; +10 monolingual languages added Jan 2026 Low-latency streaming (reported sub-300ms) ~$0.0077/min (Nova-3 Monolingual, PAYG)
Common-3 Professional Streaming AssemblyAI Closed API (EU endpoint accessible) Sure 6 languages: English, Spanish, French, German, Italian, Portuguese Sub-300ms (official); first partial ~750ms Utilization-based (PAYG)
Scribe v2 Realtime ElevenLabs Closed API Sure 90+ languages (99 per ElevenLabs) ~150ms (p50) ~$0.28/hour
Ursa / streaming Speechmatics API + on-premise + edge Sure — streaming + batch 50+ languages with automated identification Extremely-low latency (positioned) Enterprise/utilization

Advantageous-Tuning Outcomes

As a result of the weights are open, groups can fine-tune for a language, area, or accent. NVIDIA revealed a labored instance on Greek and Bulgarian. It fine-tuned the bottom checkpoint with the identical Cache-Conscious FastConformer-RNNT recipe. Every clip carried a target_lang tag for language conditioning. Coaching information got here from public corpora, together with Granary, Widespread Voice, and FLEURS.

Outcomes had been measured as WER on held-out FLEURS, on the 80ms setting. Greek WER fell from 35 to 24, a 32% relative enchancment. Bulgarian fell from 22 to fifteen, a 31% relative enchancment. These are uncooked WER percentages on the lowest-latency streaming mode. NVIDIA notes that evaluating at deployment latency, on held-out information, provides trustworthy numbers.

Strengths and Issues

Strengths:

  • One 600M-parameter checkpoint covers 40 language-locales, chopping deployment sprawl.
  • Cache-aware streaming processes every body as soon as, reported at 17x buffered concurrency on an H100.
  • att_context_size tunes latency from 80ms to 1.12s at inference, with no retraining.
  • Punctuation, capitalization, and auto language tagging are in-built.
  • Open weights enabled a 31–32% relative WER drop on Greek and Bulgarian after fine-tuning.

Issues:

  • The mannequin handles English, however NVIDIA recommends its devoted English mannequin for English-only use.
  • The 80ms mode trades some accuracy for the bottom latency.
  • Japanese and Korean use CER, so cross-language error comparisons want care.
  • Throughput figures are measured on H100, so outcomes on different GPUs will differ.
  • The manufacturing NIM with gRPC streaming is introduced, however not but launched.

Key Takeaways

  • NVIDIA’s Nemotron 3.5 ASR is an open-weights (OpenMDW-1.1), 600M-parameter streaming mannequin transcribing 40 language-locales from one checkpoint.
  • Its Cache-Conscious FastConformer-RNNT design processes every audio body as soon as, reported at 17x the concurrent streams of buffered approaches on an H100.
  • Latency is configurable from 80ms to 1.12s at inference by way of att_context_size, with no retraining.
  • A brief fine-tune minimize FLEURS WER 32% on Greek (35→24) and 31% on Bulgarian (22→15), on the 80ms setting.
  • It’s self-hostable and streaming-native, not like closed APIs (Deepgram, AssemblyAI, ElevenLabs) or offline Whisper.

Marktechpost’s Visible Explainer


NEMOTRON 3.5 ASR
1 / 10

NVIDIA · STREAMING SPEECH AI · OPEN WEIGHTS

Nemotron 3.5 ASR

A 600M-parameter cache-aware streaming mannequin that transcribes 40 language-locales in actual time, from a single checkpoint.

600M parameters
40 language-locales
80ms–1.12s latency
OpenMDW-1.1

01 — WHAT IT IS

One mannequin, 40 language-locales

  • Extends nvidia/nemotron-speech-streaming-en-0.6b with prompt-based language-ID conditioning.
  • A single 600M-parameter checkpoint covers 40 language-locales. No model-swapping.
  • Punctuation and capitalization are in-built. No separate post-processing step.
  • Targets two workloads: low-latency streaming and high-throughput batch.
  • NVIDIA nonetheless recommends its English-only mannequin for English-only use.

02 — ARCHITECTURE

Cache-Conscious FastConformer-RNNT

  • A 24-layer FastConformer encoder paired with an RNNT decoder.
  • Buffered streaming re-processes overlapping audio home windows at each step.
  • This mannequin caches encoder self-attention and convolution states, then reuses them.
  • Every audio body is processed precisely as soon as, with no overlap.
  • Compute and end-to-end latency drop, with no accuracy penalty.

03 — THE LATENCY KNOB

One setting tunes latency vs. accuracy

att_context_size Chunk (latency) Use case
[56,0] 80ms (Extremely-Low) Extremely low latency voice brokers
[56,1] 160ms (Low) Interactive voice brokers
[56,3] 320ms (Balanced) Conversational AI, reside caption
[56,6] 560ms (Medium) Larger accuracy, affordable latency
[56,13] 1.12s (Excessive) Highest accuracy

Identical checkpoint, chosen at inference time. No retraining required.

04 — LANGUAGES & DETECTION

Protection and automated language ID

  • 40 language-locales, together with English, Spanish, German, and French variants.
  • Additionally covers Arabic, Japanese, Korean, Mandarin, Hindi, and Thai.
  • Set target_lang to a recognized locale for the most effective accuracy.
  • Set target_lang=auto to let the mannequin detect the language.
  • In auto mode, it emits a language tag after terminal punctuation.
  • One deployment handles mixed-language site visitors, with no separate language-ID part.

05 — THROUGHPUT

Half the scale, extra concurrent streams

  • NVIDIA compares it towards Parakeet RNNT 1.1B multilingual, which makes use of buffered streaming.
  • Nemotron 3.5 ASR is roughly half the scale: 0.6B versus 1.1B.
  • The staff stories 17x the concurrent streams of buffered approaches, on the identical H100.
  • Avoiding redundant recomputation lowers the fee per stream in manufacturing.

The 17x determine is from the discharge announcement; the mannequin card states the qualitative declare instantly.

06 — FINE-TUNING RESULTS

A brief fine-tune lifts weaker languages

Language Base WER Advantageous-tuned Relative
Greek 35 24 32%
Bulgarian 22 15 31%

Uncooked WER (%) on held-out FLEURS on the 80ms setting. Knowledge: Granary, Widespread Voice, FLEURS.

07 — AVAILABILITY & ACCESS

Open weights, plus a hosted path

  • Weights on Hugging Face underneath the OpenMDW-1.1 license.
  • Runtime is NeMo 26.06 or newer. Enter have to be mono-channel.
  • Hosted on DeepInfra, which provides phrase boosting for area vocabulary.
  • NVIDIA says a NIM launch is deliberate for later within the month, with gRPC streaming.
  • Acknowledged GPU assist: Ampere, Hopper, Blackwell, Lovelace, Turing, Volta, and Jetson.

08 — HOW IT COMPARES

The place it sits within the panorama

Product Entry Streaming Languages
Nemotron 3.5 ASR Open weights Native 40 locales
Whisper large-v3 Open weights No (batch) ~99
Deepgram Nova-3 API / on-prem Native Multilingual
AssemblyAI U-3 Professional API Native 6
ElevenLabs Scribe v2 API Native 90+
Google Chirp / Azure API Native 100+ / 140+

Latency and WER usually are not instantly comparable throughout distributors; this compares construction, not a rating.

09 — KEY TAKEAWAYS

The quick model

  • An open-weights 600M streaming mannequin transcribing 40 language-locales from one checkpoint.
  • Cache-aware design processes every body as soon as; reported 17x buffered concurrency on an H100.
  • Latency configurable from 80ms to 1.12s at inference, with no retraining.
  • A brief fine-tune minimize FLEURS WER 32% (Greek) and 31% (Bulgarian).
  • Self-hostable and streaming-native, not like closed APIs or offline Whisper.


Curated for AI engineers by Marktechpost — practitioner-first protection of AI & ML.


Take a look at the Model weights. Additionally, be at liberty to observe us on Twitter and don’t neglect to hitch our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Have to associate with us for selling your GitHub Repo OR Hugging Face Web page OR Product Launch OR Webinar and so forth.? Connect with us


ar met nvidia Streaming
Share. Facebook Twitter LinkedIn Email
Avatar
Gavin Wallace

Related Posts

A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels

22/11/2026

OpenAI Ships GPT-Stay-1.1 for Voice Brokers, Cuts Idle Value to $0.02/Min

21/11/2026

Meet Pondering Machines Inkling 2 Mini: 26B Looped MoE Runs on a 32GB Mac

21/11/2026

Baseten Releases Bison-1 and Groundhog-1, Topping Choice Mannequin Index

21/11/2026
Top News

Join the tech reporters who are using AI to write, edit and create their stories

OpenClaw Is a Real Body

The Only Thing Standing Between Humanity and AI Apocalypse Is … Claude?

It’s all over with the fake AI about the Iran war.

Big Tech’s White House Data Center Signs Pledge with Good Optical and Little Substance

Load More
AI-Trends.Today

Your daily source of AI news and trends. Stay up to date with everything AI and automation!

X (Twitter) Instagram
Top Insights

Quantinuum Puts a 98-Qubit Helios Machine Inside Oracle’s AI Data Centers – Unite.AI

11/08/2026

Why It Works and Easy methods to Use It

02/06/2025
Latest News

Potential Cognition AI Spherical Would Elevate $2B-Plus at $75B Valuation – Unite.AI

22/11/2026

OpenRouter Cuts Three Suppliers After AI Immediate Publicity Findings – Unite.AI

22/11/2026
X (Twitter) Instagram
  • Privacy Policy
  • Contact Us
  • Terms and Conditions
© 2026 AI-Trends.Today

Type above and press Enter to search. Press Esc to cancel.