Nanbeige4.3-33B Got Its First Outside Number: 20, Level With Qwen3.8-27B, and Twice as Chatty

For two weeks the only evidence that Nanbeige4.3-33B was good at anything came from a twenty-row table the lab wrote about itself. On 5 November that changed: Artificial Analysis put Nanbeige4.3-33B on its Intelligence Index v5.0.1 at 20, the same score it gives Qwen3.8-27B at its highest reasoning setting — 20 on the composite, 20.1 for the Qwen model on the underlying v5.0 run. A tenth of a point is smaller than the noise between two evaluations of the same checkpoint, so the two are a dead heat on the only public composite that measures both.

That is the number the release was waiting for, and it is the least interesting thing in the measurement. The composite conceals a model whose two halves point in opposite directions: an agentic half that is far ahead of the Qwen model it was built to beat, and a knowledge half that is measurably behind it. It also conceals the fact that Nanbeige4.3-33B spent 160 million output tokens getting to that 20 — more than double what the median model in its comparison class burns, and enough to rank it dead last of eight on Artificial Analysis's verbosity measure.

So the honest reading of this release is no longer "unverified." It is "verified, and stranger than the vendor table suggested." Here is what the independent run actually says, where it agrees with Nanbeige's own numbers, and the one place where outside measurement came in higher than the lab's own claim.

What 20 means, and what it does not

Artificial Analysis places Nanbeige4.3-33B first of nine in its own comparison class, against a class median of 10. That class is the nine similar-size open-weight models it plots against, and the site separately says it carries 144 models in the wider comparable cohort. Both populations are worth holding onto at once: against open-weight models of roughly its size it is the strongest thing measured, and against the entire field — including frontier closed models and the best of Qwen, DeepSeek and Anthropic — a score of 20 is a solid mid-table result.

That gap between #1-of-9 and a mid-table score is worth holding onto, because it is exactly the gap that gets lost when a release is summarised as "beats Qwen." A 33B open-weights model landing mid-table on a composite dominated by models several times its size is a genuine achievement. It is not a frontier result, and nothing in the vocabulary of this release should imply otherwise.

What the index does not include is price. There is no price, because there is no hosted endpoint: Artificial Analysis records "No publicly available API," the Hugging Face inference-provider mapping for the repository is empty, and every tracker that carries the model shows cost as unavailable. Your cost is the hardware.

The two halves of the model disagree

Artificial Analysis does not publish a single number, and the sub-scores are where this model becomes legible. Read them as two columns.

• Terminal-Bench 4.0 (Artificial Analysis harness) — Nanbeige4.3-33B 29.8%, ranked 56th of every configuration on the board. Qwen3.8-27B at xhigh scores 5.6% on the same harness.

• FrontierCode 2.0 — Nanbeige4.3-33B 10.7%, ranked 81st. Qwen3.8-27B at xhigh scores 14.9%, so the code-quality benchmark goes the other way.

• AA-Omniscience v1.1 — Nanbeige4.3-33B nets −4.4 with 11.7% accuracy. Qwen3.8-27B at xhigh nets −6.4 with 17.3% accuracy. Qwen knows more; Nanbeige nets out better because it declines to answer more often.

• AA-Omniscience non-hallucination — Nanbeige4.3-33B 81.8%, ranked 7th on the entire site. Qwen3.8-27B at xhigh scores 71.4%.

• AutomationBench — Nanbeige4.3-33B 53.1%, ranked 82nd. AA-Briefcase v1.1 — 1,162, ranked 90th. Terminal-Bench-Science 0.1 — 0.9%, ranked 89th.

• HLE-Diamond — 6.9%, ranked 117th. CritPt v2 hard subset — 15.5%, ranked 85th. GDP.pdf — 11.2%, ranked 135th.

The pattern across those rows is consistent. Where the task is "drive a terminal for a long time and do not stop early," this model is roughly five times the Qwen model's score. Where the task is "know a fact," it is behind — and where the task is "know a fact or say you don't," it is ahead of almost everything on the board, at 81.8% non-hallucination.

That combination is not an accident of tuning. It is what a model built to generate agent trajectories looks like when you point a knowledge benchmark at it. A teacher model that invents facts is a teacher that poisons the student's training data, so calibration matters more than recall in the pipeline Nanbeige says it is building. Whether that trade was deliberate or emergent, it is the most defensible thing about the release.

The verbosity problem is a cost problem in disguise

Artificial Analysis reports that Nanbeige4.3-33B generated 160 million output tokens while being evaluated on the Intelligence Index. The median model in its comparison class generated 77.7 million. On the site's verbosity ranking it comes last, #8 of 8 in that class, at roughly 2.06 times the median. Input volume was 15.4 billion tokens, which puts the output-to-input ratio at 1.034%.

Two things follow. The first is that the 20 was expensive to buy. If you are paying per token anywhere, a model that emits twice the tokens for the same composite score costs roughly twice as much per unit of measured capability, and the index does not normalise for that. The second is that verbosity is a latency problem before it is a money problem: a 33B dense model on your own GPUs producing twice the tokens takes twice as long to finish a task.

Nanbeige's own recommended settings explain part of it and do not excuse all of it. The card asks for a 262,144-token ceiling for reasoning and chat, 65,536 for agentic work, with thinking on by default. Any model told it may think for a quarter of a million tokens will use some of them. But the class median is a fair comparison — other models behind the same index are also reasoning models — and the 2.06× multiple survives that control.

For a self-hosted deployment the practical fix is a budget, not a model change: cap max new tokens per task, and measure your own pass rate at 8K, 16K and 32K before assuming the card's defaults are the right ones for your workload. Most agentic tasks do not improve past a point, and this model will not tell you where that point is.

The one row where the outside run beat the vendor's own number

Nanbeige's model card claims Terminal-Bench 4.0 at 23.8 for Nanbeige4.3-33B against 6.3 for Qwen3.8-27B. Artificial Analysis measured 29.8. The independent number is six points higher than the one the lab published about itself.

That is unusual enough to be worth stating plainly, because the standard failure mode in this industry runs the other way — labs report the best number their own scaffold produces, and outside reruns come in lower. A vendor under-reporting its own agentic score suggests either a different harness on their side, a deliberately conservative claim, or a table built before the final checkpoint. None of those is bad; all of them mean the vendor table is not a ceiling and should not be read as one.

It does not follow that every vendor row is trustworthy. The largest claims in Nanbeige's table — DeepSWE at 65.5 against Qwen's 42.5, SWE-Atlas-QnA at 50.8 against 41.9 — have no independent counterpart yet, and both are scaffold-sensitive benchmarks where harness choice moves scores more than model quality does. What the Terminal-Bench result establishes is narrower and more useful: the agentic direction of the claim is real. The magnitude on any individual row is still unverified.

Where this leaves a team that wants to try it

Nothing has changed about access. There is no endpoint to call, no rate card, and no provider listing — the weights are on Hugging Face under Apache 2.0 and the serving path is vLLM, SGLang or Transformers on hardware you own. The bfloat16 checkpoint is 66.3 GB across fourteen shards, and at 512K context the KV cache alone runs to roughly 104 GiB, which is why the working configuration is four-way tensor parallelism rather than one big card.

What has changed is that you can now skip the expensive part of due diligence. Before this week, evaluating Nanbeige4.3-33B meant provisioning four GPUs to find out whether the agentic claims survived contact with a neutral harness. The 29.8 on Terminal-Bench 4.0 answers the first question — the direction is right — and the verbosity figure answers a question nobody was asking, which is what the model will cost you in tokens per completed task.

If you want a comparison point you can actually call today, Qwen3.8-27B is the natural control: same composite score, published price, and a 262K window. It runs on OrcaRouter at $0.33 per million input and $2.40 per million output, on the same key as the rest of the catalogue, with provider list prices passed through at no markup. Running your task set against a hosted model and a self-hosted one behind one routing rule is the cheapest way to find out whether a 33B dense checkpoint on your own hardware beats a 27B you rent by the token — and it is a question the Intelligence Index cannot answer for you, because it does not know what your tasks are or what your GPUs cost.

The things still genuinely unknown are narrow. Whether Nanbeige4.3-3B, the distillation target this model exists to teach, inherits any of the calibration. Whether the verbosity survives the student, since a small model that emits 2× tokens is a much cheaper mistake. And whether the non-hallucination result holds outside the Omniscience harness, because 81.8% is a remarkable number and remarkable numbers deserve a second look before anyone builds a pipeline on them.