NVIDIA Nemotron Labs Dory 1.0 4B vs MiniCPM5 2B: A Custom Runtime Against a Plain LlamaForCausalLM

The most useful thing to notice about N​VIDIA N​emotron Labs Dory 1.0 4B and MiniCPM5-2B is not either model's benchmark table — it is what appears in the first line of MiniCPM5-2B's architecture section. Its card says, flatly, that the architecture is a standard LlamaForCausalLM: 42 layers, grouped-query attention with 16 query heads and 2 KV heads, no custom kernels, no model-code fork needed. Dory's card says the opposite in every practical sense. Its architecture is a looped latent-recurrent network whose layer block is applied a variable number of times, and to serve it today you install vLLM from an open pull request or build llama.cpp from a pinned commit. One of these models drops into the stack you already have. The other requires you to change the stack first. That difference will decide more deployments than any score either vendor published.

Two small models, two completely different bets

N​VIDIA N​emotron Labs Dory 1.0 4B is N​VIDIA's 4B dense looped model, published to Hugging Face in BF16, MXFP8, NVFP4 and Q5_K_M GGUF under the OpenMDW 1.1 licence, with its card stamped "GA (10/16/2026)". It is text-only and English-only, holds up to 256K tokens, and put roughly 6 trillion pre-training tokens into a stack trained natively in NVFP4 for the first ~4.5T before switching to BF16, then extended to long context over 300B tokens and finished with SFT plus multi-environment GRPO reinforcement learning. It was published without an announcement of any kind.

MiniCPM5-2B is OpenBMB's 2.5B dense model — 2,516,756,480 parameters total, 1,981,982,720 of them non-embedding — under Apache 2.0, with a 131,072-token context window and bilingual English and Chinese support. It is the second entry in the MiniCPM5 line after the 1B, built to scale up what the card calls the same training recipe, and it is explicitly aimed at on-device, local and resource-limited deployment. Post-training ran supervised fine-tuning, reinforcement learning and on-policy distillation. The card claims 2B-class open-source state of the art on a 53.9 average across its benchmark set.

Both are small enough to run on one accelerator. Both are downloadable today. Everything after that diverges.

The contrast, dimension by dimension

• Architecture — Dory 1.0 4B is a looped latent-recurrent normalized-G​PT with interleaved sliding-window and full attention, an 8-layer input block, a 20-layer recurrent block and an 8-layer output block. MiniCPM5-2B is a conventional dense causal transformer, 42 layers, GQA 16 heads by 2 KV heads, standard LlamaForCausalLM.

• Serving cost of entry — this is the headline. MiniCPM5-2B needs no custom kernel and no forked engine; if your stack runs L​lama-shaped models, it runs this. Dory needs vLLM built from an unmerged pull request or llama.cpp built from a pinned commit, with runtime overrides for loop count and KV-cache mode.

• Parameters — 4B dense against 2.516B total (1.98B non-embedding). Dory is roughly 60% larger, and its recurrent block is applied repeatedly, so its effective compute per token at four loops exceeds what the parameter count implies.

• Context — Dory reaches 256K tokens with 262,144 positions validated in vLLM. MiniCPM5-2B supports 131,072. Dory's window is double; whether you want to pay for it is the KV-cache question covered below.

• Languages — Dory is English only, with 43 programming languages in its pre-training mix. MiniCPM5-2B is bilingual English and Chinese. For a Chinese-language workload this dimension settles the comparison without reference to any other row.

• Licence — OpenMDW 1.1 for Dory, Apache 2.0 for MiniCPM5-2B. Both allow commercial use; Apache 2.0 is the one most legal reviews have already approved.

• Compute flexibility — Dory's depth is a runtime parameter: one loop for latency, two as the recommended default, four when agentic accuracy matters more, all from a single checkpoint. MiniCPM5-2B has fixed depth, and its performance lever is quantisation — the family ships GGUF, MLX, GPTQ and LiteRT builds plus a DSpark draft model for speculative decoding.

• Ecosystem — MiniCPM5-2B has been public since early September 2026 and has over a million downloads and a spread of quantised and derivative builds. Dory's four repositories were populated overnight and their download counters were at zero when this was written.

The numbers, and why you should not subtract them

Both cards publish benchmark tables, and the tables overlap on enough tasks to make a side-by-side look possible. It is worth doing it carefully, because a naive subtraction would be misleading in a specific and predictable way.

Read Dory at its three loop settings against MiniCPM5-2B's single reported figure:

• SciCode — 27.3 / 34.94 / 35.11 against 26.3. At two loops and above, Dory leads.

• SWE-bench Verified — 50.3 / 59.5 / 61.8 against 46.4.

• Terminal-Bench 2.1 — 12.5 / 18.1 / 22.2 against 8.6.

• GPQA Diamond — 69.19 / 72.41 / 72.16 against 70.2. Even at one loop Dory is nominally behind; at three-plus loops it edges ahead.

• HLE, text-only — 8.25 / 10.89 / 10.43 against 8.9.

• IFBench — 74.29 / 77.94 / 79 against 66.3.

• AA-LCR — 63 / 71 / 73 against 59.0.

Dory looks stronger on most of these, sometimes by a lot. Now the reason not to believe the arithmetic: these are two different harnesses measuring two different prompts sets, and neither vendor ran the other's model. N​VIDIA measured Dory on NeMo Gym and the Nemo Evaluator SDK, with per-benchmark repeat counts the card discloses — SciCode eight times, AA-LCR sixteen, several benchmarks exactly once. OpenBMB measured MiniCPM5-2B on its own setup and reported a single average of 53.9 across its suite. A delta of two or three points between those two setups is noise, not a verdict. Nothing in either card is an independent reproduction, and no third party has published a head-to-head of these two models.

There is one structural reason to expect Dory's apparent edge to be real on agentic tasks and to evaporate elsewhere. Its card shows looping buys the most on SWE-bench and Terminal-Bench, the least on knowledge recall — AA-Omniscience barely moves across loop counts. MiniCPM5-2B's own strength is spread differently: it reports a 97.1 on τ²-Bench Telecom and 88.7 on GAIA Text-103, agentic results N​VIDIA's card does not measure in a comparable form. If your task is tool use in a narrow domain, the MiniCPM card's numbers are the ones to test first, and none of them appear on Dory's sheet.

Note also what the two cards decline to compare. MiniCPM5-2B benchmarks itself against Qwen3.5-4B at 51.1, granite-4.2-3B at 42.7 and LFM2.5-2.6B at 33.2 on its own average. Dory benchmarks itself against Qwen3.5-4B, Qwen3.5-9B, Gemma 4 26B A4B and Qwen3.6-35B-A3B. The only model both cards put a number on is Qwen3.5-4B, and they score it differently — which tells you more about the harnesses than about Q​wen.

What each one costs you to operate

MiniCPM5-2B's operating story is the boring one, and boring is the compliment here. Standard architecture, no custom kernels, a spread of ready-made quantisations, and a family that already includes MLX builds for Apple Silicon, GPTQ for GPU inference, LiteRT for mobile, and a draft model for speculative decoding. A 2.5B LLaMA-shaped model at 128K context fits in modest memory and serves on the engines you already run. OpenBMB frames the whole line as on-device and resource-limited by design, and nothing in the card requires you to adopt anything new.

Dory's operating story is the interesting one and it is not finished. vLLM support sits under review in pull request 60149; until it merges, the card tells you to install vLLM from that branch with the precompiled-kernel flag set, then launch with overrides configuring the recurrent loop count and the KV-cache mode. llama.cpp support is in pull request 30306, with a pinned revision rather than the moving head, and build instructions targeting native ARM64 macOS on Apple Silicon. N​VIDIA's only published Dory speed figures are quantisation speedups on a DGX Spark — 1.69× and 1.84× for MXFP8 over BF16 depending on the input/output mix, 2.17× and 2.44× for NVFP4 — not throughput numbers. There is no hosted endpoint for Dory anywhere we can verify.

The one genuinely novel piece of engineering on the Dory side is the memory design, and it is the claim most worth testing before you commit. Standard recurrent looping keeps a separate KV cache per loop, so memory climbs with each added pass. Dory supports a last-loop KV-caching mode that holds a single cache per recurrent attention layer and overwrites it on each pass, keeping the footprint at roughly the one-loop size no matter how many loops you run. N​VIDIA says accuracy stays close to the per-loop mode with nearly unchanged output lengths. If that holds, elastic depth is a real capability; if the accuracy drifts, the feature costs more than the card implies. Nobody outside N​VIDIA has checked.

Where a router earns its keep — and where it does not

MiniCPM5-2B is not on the OrcaRouter catalogue, and neither is N​VIDIA N​emotron Labs Dory 1.0 4B. Dory has no hosted endpoint anywhere we can confirm; MiniCPM5-2B is an on-device model whose card is written for local deployment, not for an inference platform. Saying that plainly matters more than finding an angle, because the practical effect is that both sides of this comparison run on hardware you own.

That is when the hosted baseline becomes the awkward part of the experiment. Most teams evaluating a 2B or 4B model are not evaluating it in a vacuum — they are asking whether it can replace a hosted call at a fraction of the cost and a fraction of the capability, and answering that question means running the hosted model on the same task set. One OrcaRouter key reaches those hosted models at provider list price with 0% markup, no second contract and no second SDK, so the comparison does not need its own procurement. Automatic failover matters for a different reason here: a long local evaluation on a single GPU will get interrupted, and having the fallback path already wired behind the same endpoint keeps the run alive. The routing DSL lets a local endpoint and a hosted one share a single call site, so the two halves of a hybrid deployment do not diverge into two codebases. None of this picks a winner between Dory and MiniCPM5-2B. It makes the loser cheap to rule out.

The verdict

Choose MiniCPM5-2B if deployment friction matters more than peak agentic scores. It is Apache 2.0, half the size, bilingual, standard-architecture, and surrounded by quantisations including mobile runtimes. If your team has to ship this month and the alternative is fighting a kernel build, the 2.5B wins on logistics alone — and its reported τ²-Bench and GAIA results suggest it is not giving up much on the tool-use tasks small models are actually deployed for.

Choose N​VIDIA N​emotron Labs Dory 1.0 4B if the thing you want is not a model but an experiment: a single checkpoint that scales its own accuracy with inference compute, a 256K window with a caching design intended to make that affordable, and a licence that permits commercial deployment. Go in knowing that you will build your serving engine from a branch, that every published number is the vendor's own on the vendor's harness with the repeat counts disclosed in a footnote, and that the download counters are still zero. The upside is that you would be among the first people outside N​VIDIA to test a genuinely different way of spending compute on a small model. The cost is that you would be doing N​VIDIA's verification work for it.

The thing that would change this comparison fastest is not a benchmark. It is vLLM and llama.cpp merging their Dory pull requests. The moment Dory loads from a released binary, the friction argument that currently favours MiniCPM5-2B mostly disappears, and the two models finally compete on the terms the benchmark tables imply rather than on the terms your build system imposes.

None of that is hypothetical: the documented AI failures and agent incidents behind releases like this one are collected in the OrcaRouter AI Incident Archive, where each entry carries its own sources, severity, confidence grade and disputed flag.