Nemotron 3.5 Transcribe vs Whisper Large v3 Turbo: 40× the Footprint for One WER Point

If your transcription job is a folder of three-minute mono interview clips and the machine you have is a 24GB workstation card, the comparison between Nemotron 3.5 Transcribe and Whisper Large v3 Turbo is settled before accuracy enters the room. NVIDIA's new model is a 34-billion-parameter speech system that was validated on an 80GB accelerator. OpenAI's turbo variant is an 809-million-parameter model that fits in 0.8GB of int8 weights and runs on a CPU. They are being compared this week because their error rates landed within about a point of each other, and that is the least useful fact about the pairing.

Nemotron 3.5 Transcribe shipped on 2026-10-28 as open weights under the OpenMDW-1.1 licence, which permits commercial use. Whisper Large v3 Turbo is from 2024-10-01 and is MIT-licensed; it is not news, it is the baseline every new speech model gets measured against. The question this article answers is not which scores better — both vendors would like you to read it that way — but which one you can actually run, at what cost, on the audio you have.

What each model actually is

The architecture difference is not a generation gap. It is a category difference, and it explains almost every downstream trade-off.

• Architecture — a hybrid Mamba–Transformer mixture-of-experts LLM with a speech encoder and a frozen 99M diarization encoder bolted in, roughly 34B total with 4.3B active per token, versus an encoder–decoder distilled from large-v3 with its decoder cut from 32 layers to 4.

• Audio window — recordings up to one hour processed in a single pass, versus a fixed 30-second window with chunked long-form decoding on anything longer.

• Speaker labels — built into the output, up to 8 speakers, in 13 languages, versus none at all; Whisper has no speaker token in its vocabulary, so a separate diarizer is mandatory.

• Word-level timing — a lightweight CTC adapter aligns the finished transcript to the audio, versus segment-level timestamp tokens only, with word timing requiring forced alignment.

• Language coverage — 40 languages transcribed, 31 translated to or from English, 13 with speaker attribution, versus 99 languages transcribed and no translation.

• Weights on disk — roughly 68GB in BF16 under OpenMDW-1.1, versus 1.6GB at fp16, 0.81GB int8, under MIT.

The accuracy numbers do not line up, and that is the point

Artificial Analysis runs the only public index where both models appear on the same scale, and its numbers are worth reading carefully because they disagree with each other. On the leaderboard table, Whisper Large v3 Turbo is listed at 4.6% AA-WER on one fast-inference endpoint and 5.5% on a second. Nemotron 3.5 Transcribe is listed at 3.4% for a hosted endpoint. NVIDIA's own announcement quotes 2.80% AA-WER, tied for first among open-weight models, and the same leaderboard's prose section repeats 2.8% while its table carries 3.4% — a discrepancy inside a single page, before anyone starts comparing vendors.

Take the table at face value and Nemotron 3.5 Transcribe is roughly 1.2 points better. Now notice that Whisper Large v3 Turbo's own two entries differ by 0.9 points on the identical model. Most of the apparent gap between these two systems is endpoint configuration — batching, decoding settings, audio pre-processing — not architecture. Anyone quoting a single WER difference as a verdict is quoting a number that moves by nearly the same amount when you change hosts.

The Open ASR Leaderboard tells a bigger-sounding story. Whisper Large v3 Turbo's English short-form mean sits around 7.8% there, while NVIDIA reports 4.71% for its model across nine test sets on the research page, and 5.33% average English WER in the accompanying Hugging Face post. Those are different test mixtures with different normalisation, so the three-point gap is directional at best. What matters more is what does not exist: a vendor head-to-head against Whisper Large v3 Turbo. NVIDIA compares its model to cascaded diarization systems and to other open-weight speech models, not to the turbo checkpoint most teams are running today. Every "beats Whisper" figure in circulation is an inference across benchmarks, ours included.

The capability gap Whisper cannot close

Setting the prompt-driven output aside, the operative difference is that Nemotron 3.5 Transcribe produces speaker-attributed transcripts in one pass. Speaker tags come from the frozen diarization encoder fused into the audio features, the response carries raw speaker segments alongside the tagged transcript, and the model labels them as <spk:0> through <spk:7>. For a two-person podcast this is a convenience. For a 60-minute disciplinary hearing, a clinician visit or a multiparty sales call, it is the product.

Whisper Large v3 Turbo cannot express a speaker at all. Its training transcripts never carried speaker labels, and its input is downmixed to mono 16kHz, which destroys the one physical cue that might have survived. Asking it to produce labels yields text that looks right and has no basis in the audio. The standard workaround is WhisperX: forced alignment for word timing plus a separately hosted diarization model, joined at segment boundaries. It works, it is widely deployed, and it is three moving parts where Nemotron 3.5 Transcribe is one.

The same structural advantage applies to long audio. One hour in a single pass beats stitching 30-second windows, and window stitching has a known failure mode: decoding without timestamp tokens drops words at almost every boundary — one measured 20-minute talk lost roughly 3% of its words that way. Nemotron 3.5 Transcribe's CTC adapter exists precisely to avoid that class of error, though note that word timestamps were not part of the initial vLLM support.

On the tasks it was built for, NVIDIA's reported numbers are strong and vendor-reported: 18.26% DER across 21 languages on the David AI diarization benchmark, described as best of 14 systems; 32.4% cpWER on VoiceArena, also best of 14. Contextual entity lists, supplied in the prompt, are credited with a six-fold reduction in named-entity errors on ContextASR-Bench. None of this has an independent replication yet, and all of it is a claim about a model that is one day old.

Where Whisper Large v3 Turbo still wins outright

Language breadth is the first answer, and it is less boring than it sounds. Whisper Large v3 Turbo covers 99 languages. Nemotron 3.5 Transcribe covers 40 for transcription, 31 for translation, and 13 for speaker attribution. If your queue contains Kazakh, Tamil, Swahili or Welsh, the comparison ends there: the turbo model transcribes them, and the newer, larger model does not claim to.

The second answer is deployment friction. Whisper Large v3 Turbo runs under whisper.cpp, faster-whisper and MLX, on CPU, on a laptop, on a Jetson. Nemotron 3.5 Transcribe wants vLLM 0.32.0 plus the NeMo Speech plugin, or SGLang-Omni 0.2.0, on an 80GB H100 in BF16, an L40S in FP8, or an RTX 5090 in NVFP4. Those are not comparable operational commitments.

The third is that turbo's price is a published, boring per-hour rate. Hosted endpoints bill around $0.04 per audio hour with a ten-second minimum, and it runs at roughly 250× real time on a fast endpoint. A thousand hours of audio costs about $40. Nemotron 3.5 Transcribe has no disclosed list price for the full checkpoint, and the hosted endpoints that carry it do not publish per-minute rates.

Check which Nemotron 3.5 you are benchmarking

The most common error in this week's coverage is comparing the wrong model. A large share of published "Nemotron 3.5" comparisons actually evaluate nvidia/nemotron-3.5-asr-streaming-0.6b — a 0.6B cache-aware streaming model from an earlier release, with its own numbers and its own weaknesses. If a table mentions 80ms to 1120ms chunks, it is not testing Nemotron 3.5 Transcribe. That matters because the streaming model loses to Whisper Large v3 Turbo on raw accuracy in real-world comparisons (3.35% against 2.55% on one multilingual leaderboard) and loses badly on word alignment, while the model discussed here is a different, non-streaming, 34B system.

The second trap is serving. The easy path, SGLang-Omni 0.2.0, gives you a working endpoint without NeMo — and without timestamps, speaker labels or translation. The full path, vLLM 0.32.0 with the NeMo Speech plugin, gives you all three, but word-level timestamps arrived as a follow-up rather than in the first PR. Three deployment recipes called "Nemotron 3.5 Transcribe" can therefore behave like three different products. If speaker attribution is the reason you are switching, verify that path before you commit engineering time to it.

What it costs to run, honestly

The hosted comparison is almost trivial: Whisper Large v3 Turbo at roughly $0.04 per audio hour is ~$40 per thousand hours, with no hardware to own. Nemotron 3.5 Transcribe is not sold that way yet, so a team that wants it is pricing a GPU, not a bill.

NVIDIA's own throughput claim is the compute floor: 2,238 seconds of audio per second of GPU time, NVFP4 with the DFlash2 drafter on an RTX PRO 6000 Blackwell, which works out to about 100 hours of audio in under three minutes. At roughly $2 per GPU-hour on-demand, that implies a floor well under a cent per audio hour — at full batching, which is not how a production queue behaves. Treat it as the ceiling of what the hardware can do, not as a rate you will hit.

Which means the real comparison is not Whisper Large v3 Turbo against Nemotron 3.5 Transcribe. It is Nemotron 3.5 Transcribe against Whisper Large v3 Turbo plus a diarization model, a forced aligner, segment-merging code and the per-language tuning that glue requires. If you already run WhisperX in production, you have paid most of that cost and the case for switching is weaker than the WER gap suggests. If you have not built it, the 30B model is competing against a pipeline you would otherwise have to staff.

One practical note for anyone running both during a migration: keeping two transcription backends behind one OpenAI-compatible endpoint turns the cutover into a config change rather than a second vendor contract. That is the layer OrcaRouter exists to provide — one key and one endpoint across the providers it routes, with provider list price passed through at 0% markup and automatic failover between them. It does not change which model is better. It changes what it costs to find out, and what it costs to be wrong about it.

Who should pick which

• Meeting, legal, clinical or contact-centre recordings with multiple speakers and an audit trail — Nemotron 3.5 Transcribe. Speaker attribution and word timings in a single pass is the capability you would otherwise assemble, and the one-hour window removes the chunking logic entirely.

• High-volume mono audio, wide language coverage, laptop or CPU deployment, predictable per-hour billing — Whisper Large v3 Turbo. At 809M parameters and $0.04 an hour it is the cheapest correct answer for a very large share of real traffic, and it is two years battle-tested.

• Mixed workloads — run turbo over everything, and route only the audio that needs speakers, jargon lists or translation to the larger model. The selection rule is audio properties, not model prestige: number of speakers, presence of a domain glossary, and recording length.

Questions worth asking before you migrate

Can Nemotron 3.5 Transcribe run on a single 24GB GPU?

Not comfortably. NVIDIA validated BF16 on an 80GB H100, FP8 on an L40S, and the NVFP4 checkpoint on an RTX 5090, which is a 32GB card. A 24GB device is below every validated configuration, and the audio tower is copied onto every tensor-parallel rank, so multi-GPU sharding does not rescue a small single card.

Does Whisper Large v3 Turbo do speaker labels or translation?

Neither. The turbo variant was trained on transcription only, so passing a translate task produces unusable output — translation needs the full large-v3 checkpoint. Speaker labels require a separate diarizer such as pyannote, and joining its turns to Whisper's coarse segment timestamps is the part of the pipeline that takes the longest to get right.

Is the parameter gap visible as latency?

Not per minute of audio, no. Whisper Large v3 Turbo returns a 30-second window in well under a second on a fast hosted endpoint, and Nemotron 3.5 Transcribe processes a one-hour file in a single pass at throughput NVIDIA describes in the thousands of audio-seconds per GPU-second. The visible difference is elsewhere: turbo has no streaming mode and neither does Nemotron 3.5 Transcribe, so live captioning is out of scope for both.

Which one is better for non-English audio?

Neither answer is safe. Whisper Large v3 Turbo covers 99 languages with quality that varies widely by language, and Nemotron 3.5 Transcribe covers 40 with vendor-reported multilingual results and an adaptation recipe — NVIDIA reports a held-out medical domain improving from 22.44% to 12.17% WER after fine-tuning on one GPU for a few hours. There is no independent multilingual head-to-head yet, so for any language outside English the responsible move is a blind test on your own audio.

What would change this verdict

Three things, in order of likelihood. Word-level timestamps landing in vLLM's transcription API would make the full serving path the default one, and shrink the gap between the three deployment recipes. Quantized checkpoints shipping broadly would move the hardware floor down from an 80GB card toward consumer devices. And a genuinely independent multilingual head-to-head against Whisper Large v3 Turbo would replace inference with measurement.

Until one of those lands, the decision is not close to a tenth of a WER point. It is a question about your audio: if it has more than one voice in it, and you need to know which voice said what, Nemotron 3.5 Transcribe is the first open model that does that in one pass without a pipeline behind it. If it does not, an 809M model released a year ago still does the job for forty dollars a thousand hours, and the 34B model is a large amount of hardware to buy for a number that moves when you change hosts.