Granite Speech 5.0 470M TurboCTC: IBM's Apache-2.0 ASR claims 12,600 RTFx

Granite Speech 5.0 470M TurboCTC is the model in IBM's new speech drop that you are actually allowed to ship, and that is the first thing to know about it. IBM put two English speech-recognition checkpoints on Hugging Face on August 25, 2026 — Granite Speech 5.0 470M TurboCTC under Apache 2.0 and Granite Speech 5.0 470M TurboCTC-NC under a non-commercial CC-BY-NC-SA-4.0 license. Same 470-million-parameter architecture, different data, opposite licensing. The Apache model is the one IBM points at for "other use cases" — which is vendor-speak for commercial production — and it is also the one carrying the headline number: IBM reports aggregate throughput above 12,600 RTFx on a single NVIDIA H200, which it translates as more than three and a half hours of English audio transcribed per second with batched inference.

That speed figure is a vendor claim a day old and unreproduced by any third party, so read it as a strong on-paper number rather than an audited fact. The reason it is worth your attention anyway is the design behind it: Granite Speech 5.0 TurboCTC drops the language-model decoder that every previous Granite Speech model used, leaving a pure Conformer encoder trained with connectionist temporal classification (CTC) and decoded non-autoregressively. No token-by-token generation loop is how a 470M-parameter model ends up claiming throughput that beats models several times its size — and it is why this release matters for anyone building speech-to-text, not just for the benchmark table.

What actually shipped

Two checkpoints, both released August 25, 2026 on the ibm-granite Hugging Face org, both English-only ASR with the same encoder-only architecture:

• Granite Speech 5.0 470M TurboCTC — Apache 2.0, commercial use permitted, trained on roughly 60,000 hours of English audio.

• Granite Speech 5.0 470M TurboCTC-NC — CC-BY-NC-SA-4.0, research and non-commercial use only, trained on roughly 75,000 hours of English audio.

This article is about the Apache model — the one a product can legally depend on — with the -NC twin referenced where the licensing story needs it. Both checkpoints ship as safetensors in F32 and BF16, and both have native support in Hugging Face Transformers through AutoModelForCTC and AutoProcessor. The feature lands in the next Transformers release; until then you install Transformers from source, load processor and model, and run greedy decoding. IBM also links a browser-based WebGPU streaming demo, which is the fastest way to hear how low the latency actually goes.

The architecture bet: an encoder, no decoder, 12.5 tokens a second

The design change is the story behind the speed claim. Earlier Granite Speech releases paired an acoustic encoder with a projector and a language-model decoder, and that decoder is what got dropped. Granite Speech 5.0 470M TurboCTC is a 16-layer Conformer encoder trained with CTC, and inference is a single non-autoregressive greedy pass. There is nothing to sample, so there is no generation latency to wait on.

Three configuration details make it fast rather than just small:

• Temporal subsampling by a factor of 8. The front-end runs at 100 frames a second; the model outputs at 12.5 tokens a second. Stacking and skipping log-mel frames, then strided convolutions and pooled residuals in the first two Conformer blocks, cut the output rate in three 2x stages.

• A subword vocabulary instead of characters. Earlier Granite Speech encoders emitted 50 characters per second; 5.0 emits 12.5 subword tokens per second from a 16,384-unit BPE vocabulary (SentencePiece in the -NC variant). Fewer output units per second of audio means the CTC decoder has fewer decisions to make.

• Self-conditioned CTC. The middle Conformer layer refines the model's own intermediate representations, which is the trick that lets a small encoder hold its accuracy while the decoder is gone.

All of that is in the model card and the IBM technical write-up. What it adds up to is a checkpoint built for laptops, phones, and streaming pipelines where a heavyweight autoregressive decoder would not fit — the edge positioning is explicit in IBM's own description.

The numbers IBM is claiming

Everything in this section is IBM-reported, run through the Open ASR leaderboard's public harness on Hugging Face infrastructure as of August 25, 2026, and not independently reproduced. Treat them as vendor numbers that look credible, not as settled truth:

• Throughput — over 12,600 RTFx on a single NVIDIA H200 with batched inference, which IBM translates as more than 3.5 hours of audio per second. IBM adds that this is over 20x the throughput of earlier Granite Speech models. This is a datacenter-GPU, batched-inference figure, not what a laptop will give you.

• Accuracy — a 5.00% aggregate word error rate for the Apache model on the Open ASR leaderboard's public English short-form test sets (the -NC variant scores 4.85% on the same sets). Inference was run through Hugging Face's jobs infrastructure and scored with the leaderboard's tooling, so IBM expects these to match the official table once the new models are absorbed into it.

• Far-field — on the FFASR noisy-and-reverberant leaderboard, the Apache model ranks ninth in accuracy and the -NC model fifth, and the two are the fastest entries on that board (throughput measured on a single NVIDIA L4).

• Training — roughly 60,000 hours of public English audio for the Apache model, done in ten days on eight NVIDIA H100s on IBM's Blue Vela cluster.

What Apache 2.0 actually buys you

The license is where this release is unusual. Open-weight speech models tend to come out under research licenses or with obligations that make a production team's legal department wince; here, the commercially-usable twin is the one IBM scored slightly lower on accuracy. You trade 0.15 points of aggregate WER (5.00% vs 4.85%) for the right to deploy in a product, to monetize the output, to fine-tune and keep your derived weights to yourself, and to use it in cloud infrastructure without a research-only clause in the way.

That trade is easy to make in the wrong direction if you chase the leaderboard. The -NC model's 4.85% comes from roughly 15,000 extra hours of training data (GigaSpeech and SPGI Speech), and it is the version IBM literally labels "for research and noncommercial purposes only." It is a fine benchmark model and a bad product dependency. The Apache model loses a little accuracy and gains the ability to be the base of a commercial transcription pipeline, a call center QA system, or an edge device. If you are evaluating this family, the licensing decision comes before the benchmark decision.

What you give up

Dropping the language model is not free, and the model card is honest about the cuts:

• No speech translation. Earlier Granite Speech generations could pass through a language model to translate as they transcribed; 5.0 is transcription only.

• No keyword biasing. The LM also handled biasing toward domain terms and names; with it gone, you get whatever the subword vocabulary naturally produces.

• English only. There is no multilingual checkpoint in this release, and no indication one is coming soon.

• The 5.00% WER is a short-form, clean-audio figure. The FFASR result (ninth, on noisy far-field speech) is the more realistic indicator for meeting-room and hands-free use, and it is a rank, not a headline number.

For a narrow transcription job — call audio, meetings, voice memos, subtitles, dictation — none of these are blockers. They matter if you were hoping one small model would also translate, or would reliably transcribe a custom vocabulary out of the box.

How to run it today

You do not need a cloud API that does not exist yet. The model runs locally:

• Install Transformers from source (the CTC feature set is not in the latest release yet), then AutoModelForCTC plus AutoProcessor and greedy decoding — the standard pipeline. The processor can compute log-mel features on the model's device, saving a host-to-device copy.

• The model card example is a two-liner via pipeline: automatic-speech-recognition with model "ibm-granite/granite-speech-5.0-470m-turboctc".

• A WebGPU streaming demo runs in a browser tab and is the fastest way to gauge latency before you commit an afternoon to a benchmark harness.

• For production, the practical question is serving. Open ASR models in this class are typically run on CPU or a small GPU with something like batched streaming inference — the 12,600 RTFx headline is an H200 batch number; a realistic single-stream laptop number will be far lower, and IBM has not published time-to-first-token figures.

That serving layer is where the real cost and complexity of open ASR live, and it is also where a routing platform earns its keep. OrcaRouter's model is one API across 200+ models with provider list price passed through at 0% markup — so when a vendor cuts a price, the cut is live the same day — plus automatic failover across providers and a routing DSL for composing several models into one call. None of that applies to Granite Speech 5.0 470M TurboCTC specifically, because neither variant is on OrcaRouter yet: the weights are a day old and no commercial provider is serving them. When an open speech model like this does get a hosted path — or when you compare it against the hosted ASR APIs already out there — a routing layer is how you try it and fall back without rewriting the integration, and the pass-through pricing is what keeps the comparison honest.

What is still unconfirmed

A one-day-old model has a short paper trail, and it is worth listing exactly what is not known yet:

• No third-party benchmark. The 12,600 RTFx, the 5.00% WER, and the FFASR rankings are all IBM's own runs through the public leaderboard harness. Nobody independent has published numbers.

• No IBM-hosted API. There is no watsonx model page, no managed endpoint, no pricing, and no date for when any of that arrives.

• No streaming latency figures. Streaming support and a WebGPU demo exist; time-to-first-token and chunk latency numbers do not.

• No same-harness comparison against the other fast open ASR models. The matchups that matter — against Whisper large-v3, NVIDIA Parakeet TDT 0.6B, and the hosted transcription APIs — have not been run on identical data by anyone yet.

What to watch next

The near-term signals are the independent reproductions. The Open ASR leaderboard is public, the harness is standard, and the weights are on Hugging Face, so a third-party run should land within days — watch whether 5.00% holds and whether the 12,600 RTFx survives on a single H200 outside IBM's own batch setup. The other thing to watch is whether IBM turns the Apache twin into a managed service, since a watsonx endpoint would instantly make this the open ASR with the most credible commercial path. Granite Speech 5.0 470M TurboCTC is, on day one, a genuinely fast, genuinely permissive English ASR with a genuinely thin verification trail. The first two are real; the third is a matter of time.