Nanbeige4.2-3B vLLM Support Is Landing: BOSS Zhipin's Looped 3B Agent Model Leaves Its Fork-Only Era

Nanbeige4.2-3B — the compact agentic model BOSS Zhipin's Nanbeige Lab released in late July 2026 — is about to become servable on a stock vLLM install for the first time. A pull request opened on September 9, 2026 adds the architecture to vLLM's model registry through the transformers backend. If it merges, vllm serve Nanbeige/Nanbeige4.2-3B becomes a stock command instead of a vendor-fork ritual. That is a real change in what a self-hoster can do: for the six weeks since the model shipped, every serving engine its model card lists — vLLM, SGLang, llama.cpp, Ollama — has pointed at a Nanbeige-maintained fork, not at an unmodified install.

The model itself is not the news; it has been downloadable since the last week of July. The news is that the fork-only era is ending, and it is ending this week. SGLang merged a native implementation of Nanbeige4.2 into its main branch on September 5, and the vLLM pull request opened four days later. Both are upstream, stock-install support for an architecture that every engine previously treated as a special case. Below, everything is labeled: what the pull requests actually do, what is verified versus still open, the vendor's benchmark claims, and the independent numbers that put them in context.

What changed this week, precisely

The vLLM pull request is vllm-project/vllm #56071, "[Model] Add support for Nanbeige4.2 (transformers backend)," opened September 9 by a Nanbeige engineer and still open at the time of writing. It is deliberately small — two files. The first adds a line to vLLM's model registry mapping the Hugging Face architecture name NanbeigeForCausalLM to TransformersForCausalLM, vLLM's generic fallback that runs a model through the transformers backend. The second adds a NanbeigeModelArchConfigConvertor, whose single job is to tell vLLM how many layers to budget for: it returns the config's num_hidden_layers multiplied by num_loops, because Nanbeige's looped transformer repeats its layer stack and vLLM must size its KV-cache and attention instances accordingly.

Two details from the review thread matter for anyone tracking this. First, the registry mapping is what makes the model run through the transformers backend automatically — a vLLM maintainer noted that once the mapping exists, the explicit --model-impl transformers flag becomes redundant, and that the remaining work before merge is a docs entry and a CI test-registry mapping. Second, reviewers flagged that a recently merged vLLM change (PR #54941) may already make the layer-count convertor unnecessary, by detecting attention modules directly instead of inferring them from a layer count. In plain terms: the fix may get simpler before it lands, not more complicated.

The vLLM path matters more because of what it is not. This is not the first attempt to put Nanbeige4.2 into vLLM natively. PR #49433, opened by the same engineer in late July as a day-zero native implementation, was closed on September 9 — the same day the transformers-backend PR appeared — after maintainers argued that a bespoke model implementation was more work than the architecture justified and pointed at the transformers backend instead. The takeaway for readers: upstream vLLM support is arriving by the compatibility route, not by a hand-tuned native implementation, and that distinction has real performance consequences discussed below.

The model that needed all this special handling

To understand why Nanbeige4.2-3B broke every runtime's assumptions, it helps to know what the model is. It is a roughly 4-billion-parameter model with 3 billion non-embedding parameters, released under Apache-2.0 in English and Chinese, aimed squarely at agentic workloads: code agents, office automation, tool use, terminal operation. The technical report (arXiv 2607.22083, dated July 24, 2026) describes pretraining from scratch on 28 trillion tokens followed by an SFT-plus-three-stage-RL recipe built around real-world environment interaction. The context window runs to 262,144 tokens. All of that is checkable.

The part that is not ordinary is the architecture. Nanbeige4.2-3B uses a "Looped Transformer": the same stack of 22 transformer layers is run twice, so a 3B-parameter model does roughly twice the per-token compute of a conventional 3B while adding no weights. That is how the lab squares a small parameter count with benchmark claims that reach above its weight class — the model effectively gets a second pass over its own representations, and the config expresses that reuse as a num_loops of 2 over 22 hidden layers (44 effective attention stages). The trade is that every inference engine has to be told how to handle a layer stack that is used twice: how to index attention for KV cache and CUDA graphs, how to size the cache, how to stream weights. A stock engine built around one-forward-pass transformers has no idea what to do with it, which is why the custom modeling code ships inside the repo and requires trust_remote_code=True in Hugging Face Transformers.

That custom code is also where the model's roughest edges live. An independent report (arXiv 2608.13987, mid-August) documented five bugs that prevented the released checkpoint from loading out-of-the-box in Hugging Face Transformers — among them a silently zeroed rotary-position-embedding buffer and calls to removed cache APIs — and community write-ups described workarounds like use_cache=False before the model would run at all. Those were fixable, and patched checkpoints and harnesses now circulate, but the pattern is the point: this is a clever architecture that has been paying an unusual tax in deployment friction since day one.

The numbers, vendor and independent

The headline benchmark claim, straight from the technical report, is that Nanbeige4.2-3B outperforms larger open models — Qwen3.5-9B and Gemma4-12B — across agentic evaluations. The flagship number is SWE-Bench Verified at 63.6 against Qwen3.5-9B's 53.1 and Gemma4-12B's 44.2. The report also lists GPQA-Diamond at 87.4, HMMT-Feb-2026 at 82.8, Terminal-Bench 2.0 at 44.1 and SWE-Bench Pro at 46.9. None of these has been independently reproduced on the vendor's chosen harness, and they should be read as the lab's own account of its model — the same account that the model card summarizes as sitting atop Artificial Analysis' small-model leaderboard.

The closest thing to an independent check so far comes from a different surface entirely. In an Artificial Analysis × Liquid AI on-device benchmark run on an iPhone 17 Pro and published in late August, a 4-bit build of Nanbeige4.2-3B tied for the highest average score among 33 working sub-8GB models at a 16K context (63, level with LFM2.5-2.6B and ahead of several 9B-class models), and at a 64K context it scored 65, second only to Ling 3.0 Tiny's 66. Its per-test profile was striking: best-in-field on MATH-500 (96%) and strong on function-calling (76% on BFCL), but a weak 33% non-hallucination rate on AA-Omniscience — and, decisively for real use, slow. It generated roughly 14 tokens per second and took 21.4 seconds and 4.0 GB to answer a 1,024-token prompt; under a 60-second answer cap its average score collapsed from 63 to 18. In other words: the quality that beats 9B models is real, and so is the cost of the looped architecture that produces it.

What upstream support actually buys you

Put the two upstream events together and the practical picture for a self-hoster is straightforward. If you run SGLang, Nanbeige4.2-3B is already servable from a stock install off the merged main-branch support — no fork, with the model's tool-calling and reasoning parsers wired to the same qwen3 detectors SGLang already ships. If you run vLLM, stock support is a merge away: the registry line routes the model to the transformers backend, the arch convertor sizes the cache correctly, and the qwen3 reasoning and tool-call parsers are reused, which is how the OpenAI-compatible tool-calling surface works.

The honest caveat is that vLLM's route is a compatibility path, not a tuned one. Running NanbeigeForCausalLM through TransformersForCausalLM means vLLM executes the model's own Hugging Face code inside its serving layer rather than a native implementation with custom kernels and CUDA-graph handling — the difference is exactly what SGLang chose to build natively. For a 3B model whose per-token cost is already doubled by the loop, the transformers-backend route is unlikely to be the fastest possible serving path, and the five-bug history of the underlying custom code means the path inherits whatever quirks remain in it. For agentic workloads, where correctness of tool calls and long-context behavior usually matter more than raw tokens per second, that may be an acceptable trade; for latency-sensitive chat it is worth benchmarking before you bet a production path on it. And two engines are still fork-only: llama.cpp and Ollama continue to point at Nanbeige branches, with the bundled llama.cpp server in LM Studio not yet supporting the architecture.

What to watch next

Three things would each change the picture. First, the vLLM PR needs to merge and ship in a release — watch the thread and the vLLM release notes; reviewers have already flagged that a docs entry and a CI checkpoint mapping remain before it is merge-ready. Second, watch whether the layer-count convertor survives review, since the maintainers believe PR #54941 may have made it redundant — a sign of how much of this workaround is scaffolding around the looped architecture. Third, watch the hosted-provider question: the Hugging Face card currently shows no inference provider serving the model, and we do not host it either, so today this is a self-host story. When a provider does list it, the routing side becomes routine — one API across a large model catalog with provider list prices passed through at no markup is the low-friction way to A/B a self-hosted Nanbeige4.2-3B against the hosted models it claims to beat. Until then, the milestone to note is the one that just happened: six weeks after a launch that every major runtime met with a shrug and a fork, two of them now serve Nanbeige4.2-3B from an unmodified install.