ZwZ2-4B vs ZwZ2-9B: Same Depth, Twice the Weights, a 1.8-Point Gap

The obvious assumption about a 4B model and a 9B model from the same family is that the smaller one is shallower. ZwZ2-4B and ZwZ2-9B are not. Both declare exactly 32 transformer layers. Both use 16 query heads against 4 key-value heads at a head dimension of 256. Both carry the same 248,320-token vocabulary and the same 262,144-position context ceiling, and both run the same 3:1 linear-to-full attention schedule. The 9B is not a deeper ZwZ2-4B; it is a wider one, and once you see that the rest of this comparison — including the vendor's own score gap — stops looking like a straight line.

The two checkpoints are also close in time and origin. ZwZ2-4B was created on 15 October 2026, ZwZ2-9B on 16 October, both by inclusionAI, both trained with the same pipeline on the same dataset of 20,662 multiple-choice questions, and neither has been announced. They differ in their base models — Qwen3.5-4B and Qwen3.5-9B respectively — and in the width of every block that follows from that.

The config diff, and what it implies

A layer-for-layer reading of the two configuration files turns up one genuine structural difference and several that follow mechanically from it.

• Depth — 32 layers in both, identical

• Hidden width — 2,560 for the 4B vs 4,096 for the 9B, a factor of 1.6

• Feed-forward width — intermediate size 9,216 vs 12,288

• Attention geometry — 16 query heads, 4 key-value heads, head dimension 256, in both

• Attention schedule — 24 linear layers and 8 full-attention layers, in both

• Context — 262,144 positions in both, M-RoPE at 11/11/10, partial rotary 0.25

• Vocabulary — 248,320 tokens in both, tied embeddings in the 4B and untied in the 9B

• Vision tower — 24 layers at hidden 1,024 vs 27 layers at hidden 1,152

• Multi-token prediction — zero MTP layers in both, the speculative head stripped from each base

• Parameters — 4,539,265,536 vs 9,409,813,744, a factor of 2.07

Two of those rows deserve a note. The untied embeddings in the 9B mean its input and output matrices are separate copies of a 248,320-by-4,096 table; that alone accounts for roughly two billion parameters, which is a large part of why the parameter ratio (2.07) runs ahead of the width ratio (1.6). And the vision tower grows with the language stack — 27 layers at 1,152 hidden against 24 at 1,024 — so the 9B sees the image through a wider encoder, not just a wider decoder.

The numbers: a bigger gain, a lower ceiling

inclusionAI publishes one aggregate per model: overall accuracy across 25 benchmarks in six domains, equally weighted. On the card's own figures the family reads like this.

• ZwZ2-4B — base 49.41, tuned 54.79, a gain of 5.38 points

• ZwZ2-9B — base 51.63, tuned 56.59, a gain of 4.96 points

• ZwZ2-27B — base 55.52, tuned 60.51, a gain of 4.99 points

So the 4B gains the most and finishes last. The 9B gains less and finishes 1.80 points higher. Both statements are true, and which one matters depends entirely on the question you are asking. If the question is "how much does this training recipe add", the answer is largest at 4B. If the question is "which model is more accurate", the answer is the 9B, by 1.80 points, and no amount of framing changes that.

All of these are vendor-reported numbers on a vendor-chosen benchmark set. Nobody has independently scored any member of this family, and the per-benchmark detail lives in figures rather than in text — an SVG and a PNG on each card. The 1.80-point gap is the honest summary of what is published; it is not a measurement you or I have seen reproduced.

Why the gain shrinks as the model grows

The pattern — 5.38 at 4B, 4.96 at 9B, 4.99 at 27B — is worth taking seriously rather than waving at. A training recipe that adds capability to a small model is filling a gap: the base model did not have the capacity or the training signal to attend to fine regional evidence, and 20,000 carefully filtered multiple-choice questions teach it a skill it lacked. A larger base model already has more of that capability, so the same curriculum has less to add. The marginal return on the data falls as the base gets better.

That is the charitable reading, and it is the one the numbers support. The uncharitable reading is that the benchmark suite has a ceiling or that the four-option task shape rewards small models disproportionately. Both readings are consistent with the published numbers, which is why the 1.80-point gap should be treated as a direction rather than a magnitude.

There is a third possibility that matters more for adoption: the gain may be concentrated. "Improved on every benchmark" is the card's claim, but an average over 25 equally weighted tasks can hide two large wins and twenty small ones. Without the per-benchmark table — and it is not in the text — you cannot tell whether the 4B's 5.38 is uniform or concentrated in exactly the fine-grained perception tasks the recipe targets. If it is concentrated, the small model is a specialist and the 9B is the generalist, and the average is the wrong statistic for choosing between them.

The 27B is the control experiment

The third size makes the family argument testable. ZwZ2-27B is built on Qwen3.6-27B — a different base generation from its two siblings — and it posts a 4.99-point gain, essentially the same as the 9B's 4.96 and well below the 4B's 5.38. If the recipe's value were about data quality alone, you would expect a flat gain across sizes. Instead the 4B is the outlier. That is consistent with the capacity-gap story and inconsistent with a story in which the training data is simply uniformly excellent.

The 27B also settles a practical point about the family's release. The three repositories appeared within 23 hours of each other, the 4B first and the 27B last, which is the signature of a batch push rather than a staged rollout. The dataset followed on 19 October. Downloads tell you where attention went: zero for the 4B, 147 for the 9B, seven for the 27B. Nobody has picked a favourite yet, and the middle size is the only one anyone has pulled.

Serving: 9 GB against 18.8 GB

The arithmetic here is simple and it is the strongest argument for the smaller model. ZwZ2-4B's weights are a 9.08 GB BF16 file; ZwZ2-9B's work out to about 18.8 GB at the same precision. On a 24 GB card, the 4B leaves room for a generous KV cache and a batch; the 9B leaves very little. On a 16 GB card, the 4B fits and the 9B does not fit at all without quantisation.

Neither model is served by a hosted provider — the Hugging Face pages report no inference provider for either — so both are self-hosting decisions. If you are choosing between them on hardware grounds, the 9B is a different machine class, not a slightly larger download. That is the practical meaning of the 2.07× parameter ratio, and it is decided by your card, not by the 1.80-point accuracy gap.

For teams that want the perception behaviour without owning the serving problem, the catalogue's hosted middle ground is qwen/qwen3-vl-8b-instruct at $0.18 per million input tokens and $0.70 per million output with a 131K context. It is not the same model and this piece will not pretend it is — but it is the comparison most teams actually need, and it runs on the same key as everything else OrcaRouter routes, with automatic failover if an upstream degrades.

If you are building a tiered pipeline rather than picking one checkpoint, the routing layer is the place to express it: send every image to a cheap model first, and escalate only the ones whose confidence is low to something larger. The routing DSL composes models into a single call, so the escalation is a rule in the route rather than a second integration. Neither ZwZ2 checkpoint can be the cheap tier today — they are not hosted — but the pattern is the one these two models are implicitly arguing about.

Which one to take

• Take the 4B if your card is 16 or 24 GB, your task is region-grounded visual evidence, and you want the largest measured improvement over a base model in this family.

• Take the 9B if you have the memory headroom and you want the highest accuracy anyone has published for a ZwZ2 checkpoint below 27B. The 1.80 points are not nothing.

• Take neither yet if you need a licence your legal team will sign. Neither repository declares one, and the cards defer to a licence file that is not present.

• Take neither if you need a paper. The v2 method has no public write-up; the arXiv identifier on both cards belongs to the first-generation ZwZ work on a different base family.

The genuinely useful experiment is not 4B against 9B on somebody else's 25 benchmarks. It is running both — or running the 4B against its own base — on a few hundred images from your actual workload, and seeing whether the 5.38 transfers. Until someone does that, the family's published numbers describe a recipe's behaviour on a task set, not a decision you can make about your own pipeline. The 4B is the interesting one precisely because its gain is the largest and its evidence is the thinnest.