Granite Speech 5.0 2B vs Granite Speech 5.0 470M TurboCTC: 35× the Throughput for 0.45 Points of WER

IBM released two Granite Speech 5.0 checkpoints within a day of each other, and the interesting thing about them is not that one is bigger. It is that Granite Speech 5.0 2B scores 5.06% average English WER at 570 RTFx on the Open ASR Leaderboard, while Granite Speech 5.0 470M TurboCTC scores 5.51% at 19,870 RTFx — roughly thirty-five times the throughput for well under half a point of accuracy. If you assumed the parameter count buys you something proportional, it does not. It buys you translation, keyword biasing, language-ID control and a language model that can follow a prompt. Speed is not on that list. The documented AI failures and agent incidents behind releases like this one are collected in the OrcaRouter AI Incident Archive where each entry carries its own sources, severity, confidence grade and disputed flag.

Both figures are IBM's own, measured on the same leaderboard and dated within a week of each other, which makes this one of the cleaner intra-family comparisons available — and also one where you should still assume a vendor eval run rather than an audited one. Neither model has an independent reproduction, and as of writing neither is served by any inference provider that I can find, including our own catalogue.

The same week, two different release stories

These two models are not twins with different sizes. They were released differently and they are built differently, and the release histories explain why their model cards read so differently.

• Granite Speech 5.0 2B — repository created 7 October 2026, weights committed that evening, card edited in the open for two weeks, release date stamped 20 October, announcement blog 21 October. New model, quiet start, loud finish.

• Granite Speech 5.0 470M TurboCTC — first shipped 25 August 2026 as an English-only model with a 12,600 RTFx claim. The 21 October release is an update, not a new model: the same architecture, a new 16,384-token multilingual BPE head, six added languages, and native punctuation and capitalisation.

That difference matters when you read the numbers. The 2B's 5.06% is a fresh evaluation of a new checkpoint. The 470M's 5.51% is an improvement on a figure of 5.78% published in August, and IBM's blog frames it as "better English WER despite adding six languages," which is the right way to read it — the multilingual update did not cost English accuracy, it gained some.

Two architectures, one consequence

Granite Speech 5.0 2B is a speech-language model. A 16-block Conformer CTC encoder feeds a two-layer window query transformer projector, which downsamples 10× to 10 Hz acoustic embeddings, which go into an intermediate checkpoint of granite-4.0-1b-base extended to 16k positions with YaRN. Supervised fine-tuning is LoRA on the projector and the language model jointly. Generation is autoregressive: the language model writes the transcript token by token.

Granite Speech 5.0 470M TurboCTC is an encoder. Sixteen Conformer blocks, CTC objective, a 16,384-unit multilingual BPE classification head, temporal subsampling by 8 to bring 100 Hz frames down to 12.5 Hz, greedy non-autoregressive decoding. No projector, no language model, no generation loop. The output is punctuated and capitalised because the model was trained to emit punctuation in the token stream, not because a second model added it afterwards.

The consequence is the whole comparison. A 2B model that generates tokens has a per-token latency floor no amount of batching removes entirely; a non-autoregressive encoder emits its whole output in one forward pass, so throughput scales with batch size and almost nothing else. That is how a model with a quarter of the parameters ends up thirty-five times faster rather than four times faster.

It is also why the 470M gives up things the 2B has. There is no prompt to follow, so there is no language-ID instruction and no keyword list, and no translation — a CTC head maps acoustics to tokens, it cannot be told to render German speech as English text.

The scoreboard, and where each number came from

• Parameters — Granite Speech 5.0 2B is 2B; Granite Speech 5.0 470M TurboCTC is 470M.

• English WER, Open ASR 9-set average — 5.06% vs 5.51%. Both IBM eval runs, 15–20 October 2026.

• Throughput, RTFx — 570 vs 19,870. Same leaderboard harness, same date range.

• Noisy and reverberant WER, FFASR — 14.80% (rank #4 of 42) vs 17.57% (rank #12). The 2B is more robust by roughly 2.8 points.

• Licence — Apache 2.0 for both. The TurboCTC also has a -NC sibling under CC-BY-NC-SA-4.0; check the repository name before you ship.

• Languages — six transcription languages (En, Fr, De, Es, Pt, Ja) plus translation, versus seven (adding Italian) with no translation.

• Control features — language-ID prompting, keyword biasing to 2,000 terms and speech translation on the 2B; none on the 470M.

• Training — 390,000 hours over 40 days on 8 H100s for the 2B; 153,000 hours over 24 days on 8 H100s for the 470M.

The robustness gap is the row people skip. Thirty-five times the speed is eye-catching, but 17.57% WER on far-field noisy audio against 14.80% is a 19% relative error difference, and noisy audio is what most real deployments have. If your audio comes from a laptop in a kitchen, the small model's speed advantage is being spent on a transcript you then have to correct.

The one place the 470M actually wins on accuracy

Japanese. The TurboCTC card claims it outperforms the 2B model on character error rate across CSJ eval1, CommonVoice-17 and FLEURS — all three test sets — and states this explicitly in the card text rather than burying it in a chart. The numbers themselves are only in a figure, so the size of the win is not quotable, but the direction is IBM's own claim about its own two models, which makes it more credible than a cross-vendor comparison.

There is a plausible mechanism. The 2B handles Japanese by adding a 92-character phonetic Katakana set to the character head and 16,500 hours of Japanese keyword-biased ASR plus 9,600 hours of Fineweb-2 T​TS to training, but it still routes the result through a language model trained mostly on European text. The 470M uses a single 16,384-unit multilingual BPE vocabulary with Japanese in it from the start and spends 8,130 hours on ReazonSpeech and 1,860 on synthetic Japanese. If your product is Japanese transcription at volume, this is the one matchup where the small model is both faster and better, and it is worth a direct test before defaulting to the 2B on size.

Cost, computed rather than quoted

IBM does not publish a price for either model. Both are Apache 2.0 weights, so the cost of running them is the cost of the hardware, and the RTFx numbers are the only lever you have for estimating it.

Take a workload of 1,000 hours of audio a day — a mid-size call-centre archive. At 570 RTFx, the 2B model needs about 6,300 seconds of GPU time, a little over 1.7 GPU-hours. At 19,870 RTFx, the 470M needs about 181 seconds, or 0.05 GPU-hours. On an H100 at roughly $2–3 an hour that is a few dollars a day against a few cents. Both are small numbers, and that is the point: at this scale neither model is expensive, and the decision should be made on accuracy and capability, not on compute.

The arithmetic changes at the other end. If you are transcribing one stream in real time on a laptop, or running an on-device app, the 2B needs a GPU to stay ahead of real time while the 470M has headroom to spare — which is why IBM positions TurboCTC for laptops, phones and edge devices and documents mlx-audio for Apple Silicon. The 2B also runs on llama.cpp and mlx-audio, but at Q8_0 with a warning that lower-bit quantisation loses accuracy past about a minute of audio.

Choosing, in the order the decision actually happens

Do not start with the benchmark. Start with what the task requires, because three of the differences are hard requirements rather than trade-offs.

Does the output need to be in a different language from the input? Then it is the 2B, and there is no comparison to make. The 470M cannot translate. This single requirement eliminates it for subtitle pipelines, multilingual meeting notes and any English-language archive built from non-English audio.

Does the transcript need to contain specific terms? Product names, drug names, ticket IDs, a customer's SKU list — the 2B accepts up to 2,000 terms as a keyword list and was trained on 23,600 hours of keyword-biased audio for exactly this. The 470M has no prompt interface. A glossary-driven deployment on the small model is not possible.

Is the audio clean? If yes, the 0.45-point English gap is small and the 35× speed advantage is close to free. If no — meetings, call centres, far-field mics — the FFASR gap of 2.8 points says the small model's errors will cost you more than its speed saves.

Is it Japanese? Test the 470M first. IBM says it wins on all three Japanese test sets.

Is it on a device? The 470M was designed for it. The 2B can run on one but was not designed for it.

The honest summary is that these are not two sizes of the same product. One is a transcribe-and-translate system with a control surface; the other is a very fast, very cheap transcription engine with no controls. Most teams that think they need "the better one" actually need to answer the translation and glossary questions first, and those two answers settle it.

Running both without betting on either

The sensible production shape for a team testing this family is to deploy both, because they have opposite failure modes. The 470M handles bulk clean transcription at a cost that rounds to nothing; the 2B handles the hard audio, the non-English audio and anything that needs a glossary. A router that classifies each incoming file — language detection, a signal-to-noise estimate, a check for whether a glossary is attached — and sends it to the right checkpoint is maybe fifty lines of code and cuts your error rate on the tail without paying 2B prices on the bulk.

That classification stage is a small model call, and it is the part worth running through one endpoint rather than embedding a second vendor SDK: OrcaRouter exposes 200+ models behind a single OpenAI-compatible API at provider list price with no markup added, and its routing DSL can express the language-and-quality dispatch as a rule instead of application code. Automatic failover matters most in the week after a release like this one, when a checkpoint is two days old, nothing is independently verified, and the safest way to try it is to keep a second path warm. None of that helps you run Granite Speech itself yet — neither checkpoint is served by us today, so the speech models stay self-hosted and the routing layer starts one step downstream.

If you only deploy one, deploy the 2B. It is the model that can do everything the 470M can do, at a speed that is still thirty-five times faster than real time and an accuracy that is better on every benchmark IBM published except Japanese. The 470M is the specialist you add when the bill or the latency profile justifies it — not the default you start from.