Nanbeige4.3-33B vs Gemma 4 31B: What a 33-Point Gap in the Vendor Table Really Means

In Nanbeige4.3-33B's own benchmark table, Gemma 4 31B scores 0.0 on Terminal-Bench 4.0. That is the row to start with, because it tells you how to read every other number in this comparison. Gemma 4 31B is Google DeepMind's 30.7-billion-parameter open-weight release, downloadable under Apache 2.0 since April 2, 2026, and Google's own materials credit it with 86.4 on τ²-bench tool use and 80.0 on LiveCodeBench v6. A model like that does not fail every terminal task in existence. A zero is what you get when a benchmark cannot tell "could not" apart from "was never handed a working tool loop."

Nanbeige4.3-33B, by contrast, is two days old. Its repository was created on October 24, 2026 at 02:49 UTC, its model card is the only announcement Nanbeige has made, and the 20-row table on that card is the sole source for every number currently circulating about it. Gemma 4 31B appears in that table as one of three comparison columns. No independent lab has run these two models against each other, and neither vendor has benchmarked the other outside this one table. So the honest version of this matchup is not a scoreboard. It is an argument about which rows survive scrutiny and which ones are measuring the harness.

What each model actually is

The two releases are six and a half months apart, and the design assumptions show it.

• Parameters — Nanbeige4.3-33B is 33.1B dense (33,118,977,024 in BF16), every parameter active on every token. Gemma 4 31B is 30.7B dense. Neither is a mixture of experts, so throughput scales with the full weight count on both sides.

• Context — Nanbeige4.3-33B ships 524,288 tokens natively. Gemma 4 31B ships 262,144. Both are unusually large for the parameter class; Nanbeige's is double.

• Modality — Nanbeige4.3-33B is text in, text out, English and Chinese. Gemma 4 31B accepts images as well as text, was pretrained on 140+ languages, and ships native function calling and a configurable thinking mode.

• Licence — both are Apache 2.0. This is the rare comparison where the licence question is settled before you start.

• Weights on disk — Nanbeige4.3-33B is about 66GB in BF16 and roughly 20GB at 4-bit. Gemma 4 31B is in the same band. Neither is a laptop model; both are single-workstation or single-node models.

The one thing that is not close is distribution. Gemma 4 31B has a managed API, an endpoint Artificial Analysis currently lists at zero cost, and independent coverage. Nanbeige4.3-33B has weights, a model card, and — in the card's own words — no inference provider deploying it. That asymmetry is not a quality judgment, but it changes what "trying it" costs.

The vendor column and Google's own card do not agree

Two rows in Nanbeige's table can be checked against Google's published figures, and only one of them matches.

• AIME 2026 — Nanbeige credits Gemma 4 31B with 89.2. Google reports 89.2. Exact match.

• GPQA Diamond — Nanbeige credits Gemma 4 31B with 91.3, and that is the single row where Gemma beats Nanbeige's own 89.5. Google's published figure for the model is 84.3. That is a seven-point discrepancy, and neither card explains the configuration behind it.

That second row deserves attention because it is the one place the vendor table hands the opponent a win. If the number came from a re-run at maximum thinking effort, it is a legitimate result and Gemma genuinely leads on graduate-level science questions. If it came from a differently configured evaluation, the table is comparing two different Gemmas. Either way, the "Gemma wins GPQA" line that has been repeated across every secondary write-up this week rests on a figure Google does not publish.

The rest of the Gemma column is harder to audit because Google reports different benchmarks. Nanbeige's table puts Gemma at 18.6 on Tau3-Bench; Google's card puts it at 86.4 on τ²-bench. Those are different versions of the benchmark family, so they are not contradictory — but they are also not a comparison, and the 68-point gap between them is a fair estimate of how much the harness and version choice can move a single model's score.

Reading the zeros correctly

Three rows in the Gemma column are near-zero: 0.0 on Terminal-Bench 4.0, 2.7 on DeepSWE, and 3.3 on SWE-Atlas-QnA. These are the rows that make the rounds on social media, and they are the least informative numbers in the table.

A near-zero on an agentic benchmark has at least three possible causes: the model cannot complete the task, the model cannot drive the tools the harness provides, or the harness was tuned for a different chat template and tool-call format. Nanbeige's card is explicit that its own results used Terminus 2 for Terminal-Bench 2.1 and mini-swe-agent v2.4.6 for Terminal-Bench 4.0, DeepSWE and SWE-Atlas-QnA. It does not say whether Gemma 4 31B was given the same scaffold with its own tool-call format, or dropped into Nanbeige's XML-oriented one. Those produce very different numbers, and only one of them is about the model.

This is not a reason to dismiss Nanbeige's result. Nanbeige4.3-33B was built around agentic coding — its card recommends an XML tool-call format, preserve_thinking=true for multi-turn tool use, and a temperature of 1.0 for agentic tasks — so it is unsurprising that it performs well inside a harness built for it. It is a reason to treat the Gemma column as a lower bound rather than a verdict.

The asymmetry benchmarks do not price: images

Gemma 4 31B takes images. Nanbeige4.3-33B does not. That single sentence decides more real deployments than any row in the table.

If your agent reads screenshots, invoices, charts, scanned forms or UI states, Nanbeige4.3-33B is not a candidate at any benchmark score. You would need a separate extraction model in front of it, with its own latency, cost and failure modes, and the pipeline's quality would then be the extraction model's quality rather than the 33B's. Gemma does that step natively, and Google's published MMMU-Pro of 76.9 and MATH-Vision of 85.6 suggest it does it competently rather than ceremonially.

Conversely, if the work is text-only — code, logs, long documents, multi-turn tool traces — the image capability is dead weight you are paying for in every forward pass. That is the case Nanbeige is built for, and it is a large share of agentic work.

512K against 256K, and what a window is for

Nanbeige's 524,288-token window is twice Gemma's 262,144, and both numbers are large enough that the difference only matters in specific shapes of work.

A 256K window holds roughly a medium-sized repository's worth of source, or several hundred pages of text. A 512K window holds twice that. For a single long document plus a long answer, both are sufficient; for an agent that accumulates tool output across hundreds of turns, the larger window buys you more turns before summarization or eviction. Nanbeige's card recommends up to 262,144 new tokens for reasoning and chat, which suggests the practical reasoning budget is closer to 256K than to the full 512K. Gemma's output ceiling is commonly configured at 131,072.

Neither vendor publishes a long-context recall benchmark here, so there is no honest way to say whose 512K or 256K is better used. Context size is a capacity claim, not a quality claim, and both cards leave it at that.

What each one costs to call

Gemma 4 31B is the only one of these two you can call as a service today. On OrcaRouter it is $0.13 per million input tokens and $0.38 per million output, with cached input at $0.02 — a pass-through of the provider's list price with no markup added, which is also why a vendor price change reaches our catalogue the same day it is announced. Artificial Analysis currently lists Google's own endpoint at $0.00 input and $0.00 output, which almost certainly reflects a promotional or free tier rather than a durable price; treat it as a trial channel, not a budget line.

Nanbeige4.3-33B has no per-token price to quote. The weights are free under Apache 2.0, and the card states that no inference provider deploys the model. Running it means four GPUs at tensor-parallel size 4 by the card's own vLLM and SGLang recipes, held open whether or not requests arrive. The marginal cost of a token is then your electricity and your utilization rate, not a rate card. Anyone comparing "$0.13 input" against "free" is comparing a bill to a capital expense.

This is also the practical argument for keeping a router in the loop even when the new model is self-hosted. The decision most teams actually face is not "replace Gemma" but "send 5% of traffic to the new thing and see what happens." Keeping the incumbent reachable through one key, with automatic failover if the experiment endpoint dies, makes that a routing change rather than a migration.

When Gemma 4 31B is simply the right answer

Four conditions settle this without a benchmark table.

• The workload includes images, PDFs or screenshots. Gemma, without argument.

• You need more than English and Chinese. Gemma was pretrained on 140+ languages; Nanbeige lists two.

• You do not want to own GPUs. Gemma has a managed endpoint today; Nanbeige has weights and a deployment recipe.

• You need a second opinion on quality. Gemma has an Artificial Analysis Intelligence Index of 15, which is modest but independent. Nanbeige has no entry there at all, so every number about it is the vendor marking its own homework.

Turn those around and the Nanbeige case is equally clear: text-only agentic work, a long tool trace, a team with GPUs, and a willingness to run its recommended XML tool format. On that shape of work, Nanbeige's own table claims a 33.7-point lead on Terminal-Bench 2.1 — 80.9 against 47.2 — and a 62.8-point lead on DeepSWE. Those margins are large enough that they probably contain a real signal even after you discount for harness advantage. They are not large enough to bet a production path on without running one yourself.

An afternoon bake-off beats this table

If you are genuinely choosing between them, the test is cheap and it does not require believing either vendor. Take twenty real tasks from your own agent logs. Run each model with its own recommended tool-call format and thinking settings — XML for Nanbeige, native function calling for Gemma — and score task completion, not benchmark resemblance. Log tokens consumed per completed task, because a dense 33B and a dense 30.7B will differ in verbosity even at identical per-token prices.

What you will learn in an afternoon is whether Nanbeige's agentic margin survives its own harness advantage. Until someone publishes that, the defensible summary is narrower than the headlines: Gemma 4 31B is the more deployable model and the only one with an independent score; Nanbeige4.3-33B is the more ambitious one, with a native 512K window and a set of self-reported agentic numbers that are striking, unaudited, and two days old.