Granite Switch 4.1 8B: One 8B Checkpoint That Chats, Decides and Transcribes

Eight billion, three hundred and eighty million parameters go in. Eight billion, seven hundred and seventy-six million come out. That 4.72% is the whole difference between IBM's base model and Granite Switch 4.1 8B, and it is the most interesting number in the release, because packed into that sliver are a safety classifier, a decision engine and a speech recogniser — all answering from one endpoint, all reading and writing the same KV cache the base model would have built anyway.

The weights landed on Hugging Face on 14 October 2026. IBM's own launch blog, Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint, followed on 15 October, which is worth stating plainly up front: this is not a quiet drop someone found in a mirror. It is an announced release that is roughly two days old at the time of writing, and the version of the story that matters is not "what did IBM claim" but "what can you actually verify from the checkpoint sitting on the Hub right now." Those two things are not the same, and this piece keeps them apart.

What the checkpoint actually contains

Granite Switch is not a new architecture. It is a composition: a frozen base model, a set of low-rank adapters that can be switched on and off per token, and — in this particular build — an encoder bolted on for audio. The composer lives in the granite-switch repository and the build manifest for this checkpoint is checked in next to the weights, which means every structural claim below is readable straight off the Hub rather than taken from a press release.

The base is Granite 4.1 8B — dense decoder-only, grouped-query attention with 32 query heads against 8 key/value heads, SwiGLU, RMSNorm, a 131,072-token window and a 100,352-token vocabulary. The composed checkpoint inherits all of that and adds two activated-LoRA adapters:

• guardian-core — rank 16, alpha 64, attached to the attention projections only (k_proj, o_proj, q_proj, v_proj), lifted from the Guardian Library release and described as detecting harm, jailbreaking, profanity, violence, sexual content, social bias and unethical behaviour.

• decision — rank 64, alpha 128, attached to all seven projections including the MLP (down_proj, gate_proj, up_proj), and trained to do something narrower and stranger: given a state and a question with labelled options, return a calibrated probability distribution over those option labels in a single forward pass.

Both are activated by control tokens — <|guardian-core|> and <|decision|> — which the chat template inserts for you when you pass an adapter_name argument. You do not write the tokens yourself, and that detail matters more than it looks: it means adapter selection is a request parameter, not a second model to deploy.

Why the parameter delta is so small

The arithmetic is the story. The base model is 8,380,551,168 parameters. The composed model is 8,776,400,896. That is +395.8 million, or +4.72%, for two adapters. The reason the number stays that low is a design decision called KV-cache normalisation: every adapter sees the base model's key/value cache rather than maintaining its own. In a conventional multi-adapter setup each additional adapter is a second set of attention state, and the memory bill grows with the number of adapters you switch between. Here it does not. IBM's claim in the launch post is that the composed model takes "approximately the same amount of VRAM on a GPU as the base model would by itself," and while nobody outside IBM has measured that yet, the 4.72% weight delta is consistent with it — and unlike the VRAM claim, the weight delta is just a number in a manifest you can read.

The decision adapter is the interesting half

Reading a probability distribution out of a language model is not new; doing it in one forward pass with a fixed vocabulary of option labels, and getting numbers calibrated enough to act on, is the part worth understanding. The invocation recipe is documented in the card: a fixed decision-engine system prompt, a State: turn evaluated on the frozen base, then a ### DECISION turn presenting the question and single-token-labelled options. You then ask the server for the logprobs of exactly those label token ids — vLLM exposes this through logprob_token_ids — and exponentiate. The documented example returns something like A: 0.982, B: 0.0035, C: 0.014.

There is a real engineering wrinkle here that the marketing version of this release skips. vLLM caps logprob_token_ids at 128 ids per request. If your decision question has more than 128 options you get an HTTP 400, and the fix is to split the label ids into chunks of 128, issue one request per chunk with an identical prompt, and concatenate — because the returned logprobs are over the full vocabulary, each label's value is independent of which chunk asked for it, so a single softmax over the complete label set is mathematically identical to an uncapped call. That is one extra request per 128 options. It is a genuinely useful detail, and it is exactly the kind of thing that only exists in the repo.

The one published number attached to this adapter is the Decision Index (JEV), edition v0.3, public fast subset of 1,068 rows: Overall 42.9, Knowledge 29.1, Language 48.9, Retrieval 49.8, Tools 59.8, Arts 18.0. Treat that as vendor-reported, because it is: IBM ran it, published it, and no third party has reproduced it. It is also worth reading the shape rather than the headline — 59.8 on Tools next to 18.0 on Arts is a very uneven profile, and the card itself warns that the option probabilities are "a prior to inform a decision, not a certified classification" on domains the adapter was not trained for.

Speech-to-text is real, and narrower than the headline

The audio path embeds Granite Speech 5.0 TurboCTC, a 470M-parameter non-autoregressive conformer CTC encoder, and transcribes incoming audio before splicing the transcript into the chat context in place of the audio content part. Calling it does not change the request shape: it is an input_audio content part on a normal chat completion, and the transcription runs on the same endpoint as chat and decisions, so there is no second speech server to stand up.

The caveats, all of them from IBM's own card rather than from a critic: the encoder is English-only, its output is lowercase and unpunctuated, there is no decode-time steering, and — the one that should stop you — the speech encoder and the LLM are not jointly trained. There is no trained projection coupling them. A mistranscription can therefore silently change what the model, or an adapter, responds to. If you are building anything where a transcription error has consequences, that sentence is the whole security review.

What is confirmed, what is vendor-reported, what nobody has checked

• Confirmed from the repo — base model identity and layer count, the two adapter ranks and their target modules, the control-token ids, the composed parameter count, the 131,072-token window, Apache-2.0 licensing, and the documented vLLM and Transformers versions the card asks for.

• Vendor-reported, unreproduced — the Decision Index scores, the claim that Granite Speech 5.0 TurboCTC is the fastest ASR model on the OpenASR leaderboard, the "same VRAM as the base model" claim, and the launch post's aside that a custom 3B Granite Switch model plays Doom at 60–100 decisions per second against a reported 10 per second for a closed decision model. IBM says it will publish more data "in the upcoming weeks," which is an admission that the current evidence is thin.

• Nobody has checked — everything else. There is no independent evaluation, no third-party leaderboard entry, no hosted API from IBM, and no published latency or throughput measurement from anyone who is not IBM. The Hub repo shows a few dozen downloads and a handful of likes: this model is two days old and largely unread.

What it costs to run, and where OrcaRouter fits

Granite Switch 4.1 8B is a self-host story right now. The card lists HuggingFace Transformers and vLLM as backends, pins a fairly narrow support window (Transformers 5.16–5.17, vLLM 0.26–0.28), and the checkpoint works with both without code changes. There is no managed endpoint to point at, which for most teams means one GPU and a container before the first request.

Being straight about the rest: Granite Switch is not in the OrcaRouter catalogue today, and neither is any other Granite model — so if you want this checkpoint, the route is the vendor's own release and your own hardware, not us. What OrcaRouter does give you is the surrounding decision. The platform routes 200-plus models behind a single key with automatic failover, and passes provider list price through with 0% markup, so when a model you are benchmarking does get a hosted endpoint, comparing it against the incumbent on your own traffic is a config change rather than a second contract. That matters most in exactly this situation: a two-day-old checkpoint with no independent evaluation is a candidate, not an incumbent, and the cheap thing to do with a candidate is keep it in a side-by-side rather than rewiring a production path around it.

The open question

Granite Switch 4.1 8B is a bet that composition beats scale for a specific class of work: applications that need to answer, decide and transcribe, where running three models means three deployments, three latencies and three bills. The engineering to make that bet cheap — shared KV cache, per-token adapter switching, control tokens resolved in the chat template — is genuinely unusual, and it is the part of this release that will still matter in a year.

What is not yet settled is whether the composed answer is good. The decision adapter's published profile is uneven, the speech path is English-only and not jointly trained, and the whole thing is two days old with no independent run behind it. Read the build manifest, run the compose command yourself, and benchmark the decision adapter on your own label set before you believe anyone's 42.9.