ZwZ2-9B vs Gemma 4 12B: One Model, One Model Card, and Which of Them You Can Actually Ship

Gemma 4 12B has been public since 23 May 2026. It ships with a model card, a technical report, a published evaluation suite, five sibling sizes, and a download count on the Hub that has crossed 79,000 in the last month alone. ZwZ2-9B has been public since 16 October 2026 — today — as eight files in a repository with no README, no licence field and no pipeline tag. Both are roughly the same size. Both accept images. Both declare the same 262,144-position context ceiling. And they are not remotely in the same state of readiness, which is the only comparison that matters if you are deciding where to spend a GPU.

The short answer: Gemma 4 12B is a finished product from Google DeepMind with five months of other people having already hit its edges. ZwZ2-9B is a checkpoint from inclusionAI that nobody outside the lab has described yet, including the lab. If you need to ship something this month, the choice is not close and it is not a commentary on the weights. If you are deciding what to prototype on next month, read the numbers below and then run your own eval, because only one of these two models has given you anything to read.

The two stacks, dimension by dimension

• Parameters — ZwZ2-9B: 9,409,813,744 in bfloat16. Gemma 4 12B: 11.95 billion.

• Architecture — qwen3_5, a hybrid of linear and full attention. Gemma: gemma4_unified, encoder-free and unified across modalities.

• Layers — 32 in ZwZ2-9B (24 linear-attention, 8 full-attention). 48 in Gemma 4 12B (40 sliding-window at 1,024 tokens, 8 full-attention).

• Context ceiling — 262,144 positions declared by both.

• Vocabulary — 248,320 vs 262,144, with Gemma's reserved for image, audio and video token ids.

• Modalities in — text, image and video for ZwZ2-9B. Text, image, audio and video for Gemma 4 12B, whose config carries a 640-dimension audio tower.

• Licence — none declared vs Apache-2.0.

• Track record — zero downloads and two likes vs 79,565 downloads and 790 likes in the last month.

Two of those lines deserve a second look because they invert an intuition. The first is the attention schedule: ZwZ2-9B is a 32-layer model with 8 full-attention layers, and Gemma 4 12B is a 48-layer model with 8 full-attention layers. Both are betting that the expensive part of attention should be rare; they simply disagree about how rare, at one layer in four versus one in six. The second is the context ceiling, where a 9.4B checkpoint with no documentation and an 11.95B flagship from DeepMind land on exactly the same 262,144 number — which is a coincidence of design targets rather than any relationship between the two.

Where Gemma 4 12B is not merely ahead, but in a different category

Audio is the first. Google's model takes sound in natively, with a dedicated audio tower and audio token ids inside its vocabulary. ZwZ2-9B's tokenizer also reserves <|audio_start|>, <|audio_end|> and <|audio_pad|> — but its config contains no audio configuration of any kind, so those tokens are vocabulary it inherited rather than a capability it has. If your workload includes speech, this is the end of the comparison, and it is worth being precise about why: not because ZwZ2-9B scored worse, but because there is nothing in the checkpoint that could process a waveform.

Documentation is the second, and it is the one that compounds. Gemma 4 12B comes with a technical report (arXiv 2607.02770), a published evaluation set covering everything from GPQA-style reasoning to vision benchmarks, a stated licence, and a note that the model is available in five sizes so you can trade capability against footprint without leaving the family. Around it, the Hub shows 83 published fine-tunes, 49 quantisations and 15 Spaces built on it. That ecosystem is not decoration. It is a body of evidence about how the model fails, produced by people who are not Google, and there is no substitute for it.

Licensing is the third, and it is the least glamorous and the most decisive. Gemma 4 12B is Apache-2.0. ZwZ2-9B declares no licence at all — not open, not custom, nothing in the repository metadata. Absent a declaration, the safe reading is that the publisher has reserved its rights, and that is not a question you want to discover you got wrong during procurement. A checkpoint you cannot legally use is not cheaper than one you can.

The one place ZwZ2-9B has a structural argument

It is not in quality, which nobody can measure yet. It is in how the thing spends compute, and the config is unusually explicit about it.

Three quarters of ZwZ2-9B's layers are linear attention — 24 of 32 — each carrying a fixed-size recurrent state rather than a key-value cache that grows with the conversation. The config names the mechanism: 16 key heads and 32 value heads at dimension 128, a 4-wide convolution kernel, and a state kept in float32 for numerical stability. Only every fourth layer does full attention. Gemma 4 12B makes a similar bet with a different mechanism — 40 of its 48 layers use a sliding window capped at 1,024 tokens — so both models are designed to stop paying full-attention cost on most layers. They differ in degree and in technique, and without benchmarks there is no honest way to say which one wins. The design intent is legible; the outcome is not measured.

Footprint is the straightforward part. 9.41 billion parameters against 11.95 billion is roughly a fifth less weight to hold before you quantise anything, and the vision tower is small — 27 layers at hidden size 1,152 projecting into 4,096 — which is why the model fits comfortably in the same class of hardware that would run Gemma 4 12B.

The behavioural difference worth knowing is the reasoning default. ZwZ2-9B's chat template emits an empty <think></think> block unless you explicitly ask for thinking, so it answers directly out of the box. Gemma 4 12B offers thinking modes you configure. For extraction, classification or routing — the jobs where a second pass of reasoning is latency you did not want to pay for — only one of these is cheap by default.

What ZwZ2-9B cannot tell you, and why that decides the comparison

You cannot know whether it is an instruction-tuned model or a base checkpoint, because its repository has no README and no pipeline tag and both would normally say so. The baked-in system prompt pointing at <answer> tags is strong evidence of a trained instruction model, but it is evidence, not a statement.

You cannot know what it was trained on, whether the fine-grained perception recipe that made the ZwZ family's name carried over, or what it is supposed to be good at. You cannot know its licence. And you cannot know how it behaves at the edges, because nobody outside inclusionAI has run it.

Put that next to Gemma 4 12B, and the asymmetry is not a matter of degree. One model arrives with a technical report and eight months of public use; the other arrives with a config.json and an inverted thinking default. That is a reason to prototype, not a reason to dismiss — undocumented checkpoints are where a lot of interesting work first appears — but it is a reason to be honest that you would be doing the evaluating yourself.

A decision rule that does not depend on the missing evidence

Take Gemma 4 12B if any of the following is true: you need audio input, you need a licence you can show a lawyer, you need to ship inside a quarter, or you need sibling sizes so the same evaluation suite transfers up and down a family. All four are settled.

Take ZwZ2-9B seriously if you have a specific question that a 9.4B hybrid-attention vision model with a no-reasoning default might answer better than one you are already paying for, and you are willing to measure it yourself. The first thing to test is not its intelligence. It is whether it accepts the images you actually send it, returns the format you actually need, and does so without a licence problem hanging over the result.

And do not compare them on a benchmark table, because the table only has one column. You could put Gemma 4 12B's published numbers beside a blank space and call it a comparison, but the blank space is the finding. Anyone presenting a head-to-head quality verdict on ZwZ2-9B this week is inventing it.

Neither checkpoint is in our catalogue today, so neither is something you can call through us right now — Gemma 4 12B is a Hugging Face download and your own serving stack, and so is ZwZ2-9B. What we do serve is the sibling that makes the Gemma comparison runnable without changing how you evaluate anything: Gemma 4 26B A4B, the mixture-of-experts member of the same family, on a hosted endpoint at $0.06 per million input tokens and $0.33 per million output tokens with the same 262K-token context. Because OrcaRouter passes provider list prices through at 0% markup, a price change on the provider side lands here the same day rather than at the next catalogue refresh — which matters when the thing you are doing is a cost comparison between a model you host and a model you rent. On the other side of the bracket, the closest hosted stand-in for what ZwZ2-9B is trying to be is qwen/qwen3-vl-8b-instruct, an 8B vision-language model with a 131K context at $0.18 and $0.70, which is documented, licensed and available now.

What would change this article

A model card. Specifically: a licence, a paragraph on training data, and one evaluation table produced by someone who did not train the model. Until then, the honest summary of ZwZ2-9B versus Gemma 4 12B is not that one is better. It is that one exists as a product and the other exists as a file listing, and the gap between those two things is the most expensive distance in machine learning.

Comparisons like this one are also where the failure modes show up: the documented AI incidents and agent failures behind models in this class are collected in the OrcaRouter AI Incident Archive, where each entry carries its own sources, severity, confidence grade and disputed flag.