
Aion-1.0-Instruct-2.3B vs MiniCPM5 2B: The One With a Scoreboard Already Has a Rival On It
- OrcaNEWOrcaCyber: Hy4 Flash Abliterated / Uncensored2026-11-18$0.07 / $0.28 per 1M tokens · 184 tok/s
- tencentNEWTencent: Hy4 Flash2026-11-16$0.02 / $0.07 per 1M tokens · 223 tok/s
- GoogleNEWGoogle: Gemini 4 Carbon2026-11-10$2.00 / $10.00 per 1M tokens · 87 tok/s
- AnthropicNEWAnthropic: Claude Fable 5.22026-11-09$8.00 / $40.00 per 1M tokens · 96 tok/s
- orcaNEWOrca: OrcaC2 1.02026-11-079 tok/s
- OrcaOrcaCyber: GLM 5.3 Abliterated / Uncensored2026-11-04$3.00 / $5.00 per 1M tokens · 90 tok/s
- OrcaOrcaCyber: Hy4 Abliterated / Uncensored2026-11-04$0.80 / $1.60 per 1M tokens · 131 tok/s
- OrcaOrcaCyber: Kimi K3.1 Pro Abliterated / Uncensored2026-11-04$2.00 / $5.00 per 1M tokens · 78 tok/s
- tencentTencent: Hy42026-11-04$0.83 / $2.50 per 1M tokens · 94 tok/s
- OrcaOrcaCyber: MiniMax M3.1 Flash Abliterated / Uncensored2026-10-29$0.80 / $1.60 per 1M tokens · 114 tok/s
- OrcaOrcaCyber: GLM 5.3 Flash Abliterated / Uncensored2026-10-27$0.80 / $1.60 per 1M tokens · 69 tok/s
- OrcaOrcaCyber: Qwen3.8 Flash Next Abliterated / Uncensored2026-10-26$0.80 / $1.60 per 1M tokens · 94 tok/s
- OrcaOrcaCyber: MiMo V2.6 Flash Abliterated / Uncensored2026-10-26$0.80 / $1.60 per 1M tokens · 41 tok/s
- MoonshotMoonshotAI: Kimi K3.1 Pro2026-10-2447Intelligence
- OrcaOrcaCyber: DeepSeek V4.1 Flash Abliterated / Uncensored2026-10-23$0.80 / $1.60 per 1M tokens · 208 tok/s
- minimaxMiniMax: M3.1 Flash2026-10-1929Intelligence
- OrcaOrca: OrcaCyber Ultra 1.02026-10-17$5.00 / $10.00 per 1M tokens · 111 tok/s
- deepseekDeepSeek: DeepSeek V4.1 Flash 10172026-10-1733Intelligence
- DeepSeekDeepSeek: DeepSeek V4.1 Flash 09102026-10-17$0.15 / $0.60 per 1M tokens · 311 tok/s
- DeepSeekDeepSeek: DeepSeek V4.1 Flash2026-10-1740Intelligence
MiniCPM5-2B's model card ships a comparison table with eight baselines in it, and Aion-1.0-Instruct-2.3B is not one of them. That is not a slight; it is a timestamp. OpenBMB's table was built around models that existed when the weights went up on 6 September 2026. Microsoft's checkpoint appeared on Hugging Face on 18 November 2026, seventy-three days later, with no announcement and three likes. Nobody has put the two side by side, because for the first day of its life nobody could. MiniCPM5-2B, meanwhile, is the most-downloaded model in this class by a wide margin — roughly 966,000 downloads against Aion's zero.
So this is not a comparison of two measured models. It is a comparison between a model that arrived with a scoreboard and a fanbase and a model that arrived with a config file and a promise about January 2027. What follows uses OpenBMB's published numbers for MiniCPM5-2B where they exist, Microsoft's card for Aion-1.0-Instruct-2.3B, and says plainly when a cell has nothing in it.
Two two-and-a-half-billion-parameter models that do not look alike
The headline sizes are close enough to invite a spec-sheet comparison, and the spec sheets then refuse to line up.
• Parameters — MiniCPM5-2B is 2.52B total, of which 1.98B are non-embedding; Aion-1.0-Instruct-2.3B is 2.29B with input and output embeddings tied.
• Depth — 42 layers for MiniCPM5-2B, on a standard LlamaForCausalLM architecture; 28 layers for Aion, on custom modelling code that requires trust_remote_code.
• Attention — 16 query heads and 2 key/value heads for MiniCPM5-2B; 24 query and 8 key/value heads at head dimension 128 for Aion, across a hybrid pattern of 21 sliding-window layers at 2,048 tokens and 7 full-attention layers.
• Context — 131,072 tokens against 32,768.
• Languages — English and Chinese for MiniCPM5-2B; eight languages for Aion, with multilingual data at about 6% of training.
• Licence — Apache 2.0 against MIT. Both permissive, both commercial-safe.
• Ecosystem — GGUF, MLX, GPTQ, base, SFT, midtrain and draft checkpoints plus the training datasets, against safetensors and a promise of quantised builds to follow.
That last line is where the gap stops being about model quality. MiniCPM5-2B is a release you can deploy on four different runtimes this afternoon. Aion-1.0-Instruct-2.3B is a release you can deploy on one, with an ONNX-WebGPU sibling repo uploaded the same evening, and a GGUF format listed on the card but absent from the file listing.
The numbers OpenBMB published, and the ones it did not
MiniCPM5-2B's card claims 2B-class open-source state of the art and then, unusually, prints the receipts across 35 benchmarks in nine categories. The headline figures: AIME 2025 at 86.5, AIME 2026 at 86.5, MMLU-Pro 70.8, GPQA-Diamond 70.2, IFEval 86.7, LiveCodeBench v6 69.1, SWE-bench Verified 46.4, BFCL v4 66.6, τ²-Bench Telecom 97.1, AA-LCR 59.0 and NoLiMa 68.1.
The card also publishes a summary average across all of them — 53.9 for MiniCPM5-2B, 51.1 for Qwen3.5-4B, 42.7 for Granite 4.2 3B, 33.2 for LFM2.5-2.6B. Those are the vendor's own harness runs on its own benchmark selection, and the selection favours exactly what the model was trained for: agentic tool use, coding, long context. A vendor that chooses the tests can choose the ranking. Read the table as a statement about training priorities, which it is honest about, rather than as an independent verdict, which it is not.
What MiniCPM5-2B's card does not contain is any benchmark where Aion-1.0-Instruct-2.3B appears, and what Aion's card does not contain is any of MiniCPM5-2B's nine categories. The overlap is real but narrow: MMLU-Pro, GPQA, IFEval and the AIME variants are the only places where the two published tables speak the same language, and even there the harness configurations differ. On those shared rows the gap is wide — 70.8 against 33.35 on MMLU-Pro, 70.2 against 28.18 on GPQA — but that is a comparison between a model trained for reasoning and agentic work and one trained for document comprehension and rewriting, and the numbers are saying exactly that.
Where Aion's honesty is the whole story
The interesting feature of Microsoft's card is not its scores. It is that the scores are unflattering and the card prints them anyway, in a table that also names its judge model, its harness, and two benchmarks whose scoring settings make them look better than the headline implies. A card written to sell something does not include a row that reads MMLU-Pro 33.35.
A 33.35 on MMLU-Pro is not a number a launch team chooses. A 28.18 on GPQA is not either. Their presence is evidence that this table came out of an evaluation pipeline rather than a marketing one, and that is a genuine reason to trust the surrounding claims — the 32,768-token window, the eight languages, the 2025-12 knowledge cutoff, the 2.29B parameter count derived from a tied-embedding configuration.
The same candour runs through the limitations. The card says outright that a model this size "can miss fine details, provide misstatements, or show inability to follow complex multi-step reasoning", that it "was not trained to excel at world knowledge", and that output quality varies across the eight languages. Compare that to a table of 35 benchmarks claiming state of the art, and you are looking at two different theories of what a model card is for. Neither is dishonest. One is a capability claim and one is a disclosure.
The on-device pitch, and the long-context trap
Microsoft's framing is unambiguous: Aion Instruct is "the foundational Small Language Model (SLM) in the Windows ecosystem", built for text and operating-system tasks on the device, with a stronger version plus a content-moderation model becoming the Windows OS SLM by the end of January 2027. The upload is described as "an introduction to what's coming".
MiniCPM5-2B's framing is broader: local assistants, coding agents, tool-use workflows and reasoning, on phones, PCs and smart cockpits, with an agent training set of 500,000 samples and 80,000-plus RL samples released alongside the weights. It is a model built to act, not just to answer.
The 131K-versus-32K context gap is where that difference shows up hardest, and MiniCPM5-2B has the receipts: AA-LCR 59.0 and NoLiMa 68.1 are long-context benchmarks, both vendor-run, both published with the visual marker its card uses for judge-scored results. Aion has a 32,768-token window that its own config suggests is reached by YaRN scaling from a 2,048-token native range, and no long-context evaluation whatsoever. If the workload involves long documents, the choice has already been made for you, and it was made by the evidence rather than by the parameter count.
Agent stacks are where the comparison changes shape
MiniCPM5-2B was trained for tool loops: 500K agent SFT samples, GRPO-style RL on verifiable tasks, and benchmark rows for SWE-bench Verified, Terminal-Bench, BFCL and a banking-domain τ³ run. Aion-1.0-Instruct-2.3B documents a tool-calling format and publishes no agentic benchmark, because it was built for summarisation, rewriting and chat on an operating system.
That asymmetry is worth thinking about architecturally rather than as a score. An agent loop is not one model call; it is a plan, a tool call, a result, a re-plan, and often a second opinion, and each step is a separate request with its own latency and failure probability. This is the layer where routing stops being an infrastructure footnote and starts being part of the design. OrcaRouter's model fusion runs a panel of models against the same request and combines their answers, and its routing DSL composes several models into a single call — the two primitives that matter when "one small model, one answer" is the wrong shape for the task. Neither Aion-1.0-Instruct-2.3B nor MiniCPM5-2B is in our catalogue today, since both are self-hosted weights rather than hosted endpoints; the argument is about the requests your local agent cannot finish alone, not about these two checkpoints.
The practical version: use the small model for the cheap steps, and make the escalation path explicit rather than hoping the 2B model gets it right on the fourth retry. A model like MiniCPM5-2B at 46.4 on SWE-bench Verified is impressive for its size and still wrong more than half the time, and no amount of prompt engineering changes the arithmetic of a five-step loop.
What you can actually verify today
This is the sentence that decides most of these comparisons. With MiniCPM5-2B you can download a GGUF, load it in llama.cpp, run the card's own benchmark commands, and check whether you reproduce 86.5 on AIME. With Aion-1.0-Instruct-2.3B you can download bfloat16 safetensors, install transformers 5.x, accept trust_remote_code=True, and run the same harness on its eleven benchmarks — which is more work but entirely possible, and nobody in public has done it.
What you cannot do with Aion is run it on the runtime most likely to be on the machine you are targeting. The card lists GGUF as a library name; the repository contains safetensors, tokenizer files, two custom modelling modules and a chat template. The ONNX-WebGPU sibling repository exists, which makes browser and DirectML deployment plausible, but a GGUF build is promised rather than delivered. If your deployment target is llama.cpp or LM Studio, that single missing file is the whole decision until it appears.
Two other things are worth checking yourself rather than trusting. First, the 32K window: fill it and see whether the tail of the document survives the sliding-window layers. Second, the tool-calling path: Aion's format wraps schemas and calls in custom tags, and how gracefully it recovers from a malformed call is not something a card can tell you.

OpenBMB's card is the only one of the two that publishes long-context and agentic results at all — AA-LCR 59.0, NoLiMa 68.1, SWE-bench Verified 46.4 — alongside a 131,072-token window and a standard Llama architecture.
Two honest use cases
Take MiniCPM5-2B when the job involves tools, code, long documents, or Chinese. Its card was written for those tasks, its benchmarks cover them, its quantisations exist, and its licence is Apache 2.0. The risk is that the state-of-the-art claim is the vendor's own comparison set, and 2-billion-parameter models in agent loops still fail in ways that cost real tokens.
Take Aion-1.0-Instruct-2.3B when the job is a Windows application summarising, rewriting or classifying local documents in one of eight languages, offline, with the data never leaving the device. It is smaller in context, weaker on every shared benchmark, and backed by a card that says so. What it offers is a documented on-device contract and a plausible path into the operating system by January 2027 — and, for now, a very small number of people who have run it.
The comparison will read differently in a month. If a GGUF appears, the runtime objection disappears. If an independent harness run appears, the numbers stop being Microsoft's alone. Neither has happened yet, and pretending otherwise would be the one thing this piece cannot do.

Microsoft's card, which prints its own weak scores and labels the loose scoring settings — the reason to trust the specification claims even where the benchmark claims are unremarkable.

The scoreboard as the evidence stands: 35 published benchmarks and four runtime formats on one side, 11 benchmarks and safetensors on the other.
