Nemotron 3.5 Transcribe vs Granite Speech 5.0 470M TurboCTC: 64× the Parameters, 43× Slower

The most useful object in this comparison is a 279-megabyte file. That is the Q4_K_M quantisation of Granite Speech 5.0 470M TurboCTC, IBM's encoder-only speech recogniser, and it will transcribe English on a laptop CPU with no GPU, no Python environment worth the name and no service contract. Nemotron 3.5 Transcribe, which NVIDIA published as open weights on 28 October 2026, is a 30-billion-parameter speech-augmented language model that wants a datacentre-class accelerator and roughly sixty gigabytes of BF16 weights. NVIDIA's model is sixty-four times larger and, on the throughput figures each vendor publishes, around forty-three times slower per GPU-second. It is also the only one of the two that can tell you who was speaking.

Neither of those facts is a knockout. They describe two different machines that happen to be measured on the same leaderboard, and the entire question is whether your workload needs the room or just the words.

A non-autoregressive CTC model against a generative speech LLM

Granite Speech 5.0 470M TurboCTC is sixteen Conformer blocks with a 16,384-piece BPE output layer, trained with a CTC objective and decoded greedily. There is no language model, no projector and no generation loop: audio goes in, a frame-synchronous label sequence comes out, subsampled from 100 Hz to 12.5 Hz. That design is why it is fast, why it is small, and why it cannot do anything the label set does not cover. It has no notion of a prompt, no output style, no translation target and no speaker field.

Nemotron 3.5 Transcribe is the opposite architecture, and the difference is not a generation gap. A speech encoder feeds a frozen Nemotron 3 Diarization model whose speaker-activity signal is injected into the encoder through a Parallel Expert Encoder, and a hybrid Mamba–Transformer mixture-of-experts decoder produces the transcript as text, with roughly 4 billion of its 30 billion parameters active per token. Because the output is generated rather than classified, it can be instructed: verbatim or written form, an expected-entity list, a translation target, speaker-attributed output, or a spoken answer to a question about the audio.

• Parameters — about 30B total, roughly 4B active per token vs 470M encoder-only.

• Decoding — autoregressive generation over a 30B MoE vs greedy non-autoregressive CTC.

• Licence — OpenMDW-1.1 vs Apache-2.0 (the more permissive of the two; IBM's separate -nc sibling is CC-BY-NC-SA-4.0).

• Footprint — roughly 60GB in BF16, NVFP4 and GGUF builds offered vs 279MB at Q4_K_M in a community GGUF port.

• Languages — 40 transcribed, 31 translated vs 7 (English, French, German, Spanish, Portuguese, Italian, Japanese) after the 21 October 2026 multilingual update.

• Speakers — up to 8, attributed, in 13 languages vs none; diarization requires a separate model.

• Timestamps — word-level via a post-decode CTC adapter vs none; CTC peak positions are not word boundaries.

• Streaming — none, recorded audio only vs none.

The accuracy gap is smaller than the size gap, and the averages do not match

IBM reports a 5.51% average word error rate across nine public English test sets for the multilingual TurboCTC release, improved from 5.78% at the August launch. NVIDIA reports 5.33% average English WER on the Hugging Face Open ASR Leaderboard, and 4.71% across nine public test sets on its own research page. Both numbers are vendor-reported, and the two averages are not the same mixture: IBM's nine sets and NVIDIA's eight-set leaderboard entry differ in composition and normalisation, and the research page's separate nine-set figure differs again from its own leaderboard entry. Do not subtract them and call the difference a quality gap. The defensible statement is narrower — on the public English short-form leaderboard these two are within a couple of tenths of a point of each other, which is a remarkable result for a model 64 times smaller and a warning about how much of a leaderboard's average is LibriSpeech-shaped clean read speech.

Word error rate also hides the thing that actually separates them. Granite is a classifier: it emits the most likely label sequence, and where the acoustics are ambiguous it emits something plausible. Nemotron generates text with a language model behind it, which is why its contextual prompting claim — a six-fold reduction in named-entity errors on ContextASR-Bench with an entity list in the prompt — is even possible. On a transcript of a sales call full of product codes, the small model has no mechanism for the bias and the large one does.

Throughput: 19,870 RTFx against 462

The Hugging Face leaderboard entry for Nemotron 3.5 Transcribe carries an RTFx of 462.09 — meaning the model processes roughly 462 seconds of audio per second of compute on the leaderboard's harness. IBM's October update reports 19,870 RTFx for TurboCTC on a single H200, up from about 12,600 at launch after a single-block graph-reuse optimisation. That is a forty-three-fold difference, and unlike the WER figures it is at least measured on two competitive accelerators rather than two different test mixtures.

NVIDIA's own headline is a different kind of number: up to 2,238 seconds of audio per second on one RTX PRO 6000 Blackwell, using the NVFP4 checkpoint and a DFlash2 draft model. That is higher than the leaderboard's 462 because it is a different measurement — 30-second clips at high concurrency, as NVIDIA's own Hugging Face post states, with speculative decoding enabled and a quantisation that was still described as forthcoming. Read the two together and the honest conclusion is that Nemotron's throughput is configuration-dependent by an order of magnitude, while TurboCTC's is a property of its architecture. A model with no autoregressive loop cannot be slow in the way a 30B MoE can.

What 470 million parameters buys you that 30 billion cannot

The first answer is deployment shape. A 279MB GGUF runs on a CPU, on a phone-adjacent edge device, inside a desktop application, on a Raspberry Pi-class board if you are patient. There are now community GGUF ports and a C++ runtime family that treats the model as a first-class engine. Nemotron 3.5 Transcribe runs under vLLM with the NeMo Speech plugin, or SGLang-Omni, on an 80GB accelerator in BF16 or a Blackwell card in NVFP4. Those are not two points on a spectrum; they are different procurement categories.

The second is latency floor. A CTC model returns a transcript in one forward pass. A generative model returns one token at a time, and its wall-clock latency on a short file is dominated by the decode loop rather than by the audio. For batch archives this is irrelevant. For anything interactive — a dictation box, a call-quality monitor, a subtitle tool that has to keep up with a scrub — the small model is the only one of the two that finishes before the user notices.

The third is licensing clarity. Apache-2.0 is the licence legal departments have already approved; OpenMDW-1.1 is new enough that some of them will want a review. If your blocker is a procurement form rather than a benchmark, IBM's model clears it faster.

What 30 billion parameters buys you that 470 million cannot

Speakers, first and decisively. TurboCTC has no speaker representation at all. To get attributed transcripts you need a diarization model in front of it, a merge step, and an error budget for the merge — the standard cascade. Nemotron 3.5 Transcribe folds a frozen diarization model into the encoder and emits speaker labels directly, up to eight of them, in 13 languages, with speaker-attributed word timing from a post-decode CTC adapter. NVIDIA's reported diarization error rate is 18.26% across 21 languages on the David AI benchmark, best of 14 systems, with the nearest competitor being its own Nemotron 3 Diarization model at 18.38%. Those are vendor numbers on a vendor harness and they have not been independently reproduced. The cascade comparison they imply — one model versus three — is not a benchmark claim at all, it is a description of the deployment.

Languages and translation second. TurboCTC covers seven languages after its October update; Nemotron covers forty for transcription and thirty-one for translation to or from English. If your queue is European and monolingual, seven is enough. If it is not, the comparison ends.

Fine-tuning third. NVIDIA reports adaptation on a single GPU at 0.5 to 2.3 GPU-hours per task, and a medical-speech example in which a fine-tune took word error rate from 30.8% to 11.8% on 3,619 clips, with held-out speakers improving from 22.44% to 12.17%. IBM's TurboCTC is also fine-tunable — it is Apache-2.0 — but you would be fine-tuning a CTC head on a much smaller model, and the ceiling on what additional context can buy you is lower because there is no language model to persuade.

The cost model that decides it

TurboCTC's economics are almost boring. One H200 serving 19,870 RTFx saturates a very large queue; a quantised 279MB build on commodity CPUs costs nearly nothing per audio hour, and the electricity is the line item. There is no per-minute meter anywhere in the stack.

Nemotron's economics are the same shape at a different scale: no per-minute meter, but a much larger accelerator and a much smaller throughput ceiling. One Blackwell card processing 2,238 seconds of audio per second covers 100 hours in under three minutes at high concurrency and considerably less on hour-long meetings with overlap.

• A 20-hour-per-month queue — TurboCTC on a CPU is effectively free; Nemotron needs a rented or owned accelerator that will sit idle most of the month.

• A 2,000-hour-per-month archive with no speaker requirements — TurboCTC on one GPU, or a small fleet of CPUs, is the cheaper and faster configuration by a wide margin.

• A 2,000-hour-per-month archive of six-person meetings — TurboCTC plus a diarizer plus a merge layer, measured against Nemotron on one card. This is the only configuration where the large model's price is defensible, and even here the cascade is cheaper on hardware.

• Anything under a confidentiality constraint that forbids uploading audio — both are self-hostable, so the constraint is met either way, and the decision falls back to speakers and languages.

The post-processing half of the pipeline is where a routing layer earns its keep regardless of which recogniser you pick. Transcripts get summarised, tagged and routed, and OrcaRouter puts 200+ text and multimodal models behind one key with provider list prices passed through at 0% markup, automatic failover, and a routing DSL that can send one class of transcript to a cheap summariser and another to a stronger model without a second integration. Neither Nemotron 3.5 Transcribe nor Granite Speech 5.0 470M TurboCTC is served on that catalogue — we do not host speech-to-text — so the speech choice stays a separate contract and only the layer above it becomes portable.

Pick by the shape of the job, not the leaderboard

If the job is "turn a lot of clean, mostly English audio into text as cheaply and quickly as possible", Granite Speech 5.0 470M TurboCTC wins and the comparison is not close — it is a 279MB Apache-licensed model that outruns a 30B system by more than forty times and gives up almost nothing on the public English average. The cascade you build for speakers is the price of that efficiency, and if you never needed speakers, you never pay it.

If the job is "produce one attributed, timed, structured transcript from a difficult recording in one of forty languages", Nemotron 3.5 Transcribe is the only one of the two that attempts it, and the vendor's diarization numbers — not yet independently reproduced — suggest it does so better than the cascades it replaces. The throughput cost is real, the hardware cost is real, and for a meeting-heavy workload neither is prohibitive.

The mistake to avoid is treating this as a generational contest. A 470M CTC model is not an outdated version of a 30B speech LLM; it is a different instrument that happens to score similarly on a benchmark built mostly from clean read speech. The next release will not make it obsolete. Your own worst recording will tell you which one you needed.