Title card for the Aion-1.0-Instruct-2.3B versus Qwen 3.8 comparison, contrasting a 2.3B on-device Windows model with a 27B open-weight vision-language model.
Guides & Insights

Aion-1.0-Instruct-2.3B vs Qwen 3.8: A Windows Component Meets a 27B Workhorse

Author

jinhao.song

Date Published

Latest models · 20View all models →
Benchmarks: Artificial Analysis · updated daily
Back to all posts

On 18 November 2026 a repository called Aion-1.0-Instruct-2.3B appeared in Microsoft's Hugging Face organisation with a single commit reading "Duplicate from asgmodels/Aion-Instruct-v1". No blog post, no model card announcement, no press release, no coverage. Its card describes it as the foundational small language model for on-device Windows work and adds that a further optimised version of it, bundled with Microsoft's content moderation model, will become the Windows OS SLM by the end of January 2027. Set that against Q​wen3.8-27B, the open-weight 27B vision-language model A​libaba published in August 2026, and the hosted Q​wen3.8-Flash and Q​wen3.8-Max tiers above it, and the comparison stops being about quality very quickly. One is a component sized for a laptop. The other is a general-purpose model you rent by the million tokens. Almost everything a benchmark table would tell you is beside the point. Q​wen3.8-27B is served as qwen/qwen3.8-27b on OrcaRouter's catalogue, on the same key and endpoint as the Q​wen3.8-Flash and Q​wen3.8-Max tiers.

What the Aion repository actually contains

Start with what is checkable, because the list is short. The repository was created at 17:05 UTC on 18 November 2026 and has not been modified since. It carries three likes and, at the time of writing, zero downloads. The weights are a single 4.58 GB BF16 safetensors file, consistent with the 2.3B name at two bytes per parameter. The vocabulary is 200,029 tokens. The licence is MIT.

The architecture is more unusual than the size suggests. Twenty-eight layers alternate in a hybrid pattern — twenty-one use sliding-window attention with a 2,048-token window and seven use full attention — and the full-attention layers carry a 16× YaRN extension. Grouped-query attention is 24 query heads to 8 key-value heads, embeddings are tied, and the card instructs you to run it with custom modelling code through trust_remote_code. The repository ships its own transformers implementation rather than mapping onto a stock architecture.

The provenance is the part that is not confirmed. The commit says the repository is a duplicate of asgmodels/Aion-Instruct-v1, and the licence link points at that repository's LICENSE file. Both asgmodels/Aion-Instruct-v1 and a sibling Microsoft repository named Aion-Instruct-Preview-Sample return an authentication error to anonymous API requests, so they are private or gated. We can see that Microsoft copied the checkpoint into a public repository. We cannot see what the original contained, who trained it, on what data, or why the organisation is named asgmodels. Anyone describing this model's training is guessing.

Screenshot of the microsoft/Aion-1.0-Instruct-2.3B model card on Hugging Face, showing the Microsoft Windows Applied Sciences Group attribution, the on-device SLM description, the January 2027 Windows OS SLM target, and the note that only the unquantised model is published first.

The numbers on the card, and what they are worth

The model card publishes eleven benchmarks. MMLU 55.15, MMLU-Pro 33.35, MMMLU 45.51, GPQA 28.18, GSM8K 74.91, GSMPlus 49.56, MATH-500 48.00, IFEval 83.90, IFBench 32.00, Multi-IF 64.70 and chrF++ 43.18. The methodology line says runs used EleutherAI's lm-evaluation-harness wherever a task implementation existed and the benchmark's own repository otherwise, with GPT-4o as judge on the tasks marked as judge-scored. Scores are percentages.

All of it is vendor-reported and none of it has been reproduced. For a model this small, though, the pattern in the table is more informative than any single figure. MMLU at 55.15 is respectable for 2.3B parameters. MMLU-Pro at 33.35 and GPQA at 28.18 are not: those are the harder, more reasoning-heavy knowledge tests, and they show the ceiling clearly. GSM8K at 74.91 with loose matching is a materially weaker result than the same number would be under strict matching. IFEval 83.90 is the strongest relative result on the card — the model follows formatted instructions well for its size, which is exactly what an operating-system assistant needs to do.

There is a second caveat that the card states plainly and that is easy to miss. The published checkpoint is the unquantised model, and the card says quantised versions will follow. The ONNX build already exists separately — Microsoft published Aion-1.0-Instruct-2.3B-ONNX-WebGPU about three minutes after the BF16 repository, at roughly 1.3 GB, using INT4 block-wise quantisation-aware training at block size 32 on linear layers and block size 64 on embeddings, with FP16 activations and KV cache, targeting WebGPU through ONNX Runtime GenAI. That is the deployment artefact that matters, and it is a different file from the one most people will download first.

Why Q​wen3.8-27B is a different kind of object

Q​wen3.8-27B is 27.8 billion parameters across eighteen BF16 shards totalling 55.6 GB, with a vision tower for image and video input. Its sixty-four layers are laid out as sixteen repeating blocks of three Gated DeltaNet linear-attention layers to one full Gated Attention layer, hidden size 5,120, with 48 value heads in the linear-attention path against 16 query-key heads. Context is 262,144 tokens natively and 1,000,000 with YaRN. Thinking is on by default, with up to 262,144 reasoning tokens and 131,072 answer tokens, and the recommended sampling differs between thinking and non-thinking modes. The licence is Apache 2.0.

The gap that matters is not parameters. It is that a 55.6 GB checkpoint does not run on the device the Aion model is built for. The Aion model's whole design — 2,048-token sliding windows, INT4 quantisation-aware training, an ONNX WebGPU build at 1.3 GB — is an argument about what fits in memory on a machine somebody already owns. Q​wen3.8-27B's design assumes a serving cluster. You do not choose between them on a scoreboard. You choose because one of them is not a candidate for the other's job.

Where the numbers can and cannot be laid side by side

There is one genuinely useful cross-reference here, and it is not on either vendor's card. Artificial Analysis measures Q​wen3.8-27B on Intelligence Index v5.0.1, which combines nine evaluations: AA-Briefcase v1.1, AutomationBench-AA, Terminal-Bench 4.0, FrontierCode-AA 2.0, AA-Omniscience v1.1, GDP.pdf, HLE-Diamond, CritPt v2 and Terminal-Bench-Sci. On that index Q​wen3.8-27B scores 20, against a median of 10 for comparable models. The same measurement puts it at $0.50 per million input tokens and $3.00 per million output, $3.53 per task, 38.7 minutes per index task against a median of 11.5, and 128 million tokens consumed across the index against a median of 77.8 million. It is slower and more verbose than the field, and it pays for that in cost per task even at a modest per-token price.

Comparison scoreboard for Aion-1.0-Instruct-2.3B and Qwen3.8-27B, contrasting parameters, context window, modality, licence, footprint and evaluation status.

There is no equivalent for Aion. As of 19 November 2026 Artificial Analysis has no entry for any Aion model, and no independent harness has published a run. So the honest scoreboard has a hole in it, and the hole is on the newer model's side.

• Parameters — Aion-1.0-Instruct-2.3B: roughly 2.3B, one 4.58 GB BF16 file. Q​wen3.8-27B: 27.8B, eighteen shards, 55.6 GB BF16.

• Context — Aion-1.0-Instruct-2.3B: 32,768 tokens, according to the ONNX build's card; the BF16 card does not state a figure. Q​wen3.8-27B: 262,144 native, 1,000,000 with YaRN.

• Modality — Aion-1.0-Instruct-2.3B: text only. Q​wen3.8-27B: text, image and video in, text out.

• Licence — Aion-1.0-Instruct-2.3B: MIT. Q​wen3.8-27B: Apache 2.0. Both permit commercial use; MIT is the less encumbered of the two.

• Runnable footprint — Aion-1.0-Instruct-2.3B: 1.3 GB INT4 ONNX for WebGPU, or 4.58 GB BF16. Q​wen3.8-27B: 55.6 GB BF16, plus serving memory.

• Independent evaluation — Aion-1.0-Instruct-2.3B: none. Q​wen3.8-27B: Intelligence Index 20 on v5.0.1, plus a vendor card reporting LiveCodeBench v6 90.3, GPQA Diamond 89.2, Terminal-Bench 2.1 at 73.0, SWE-bench Pro 61.7 and HLE 30.8.

• Publication date — Aion-1.0-Instruct-2.3B: 18 November 2026, one day old and unannounced. Q​wen3.8-27B: announced 3 August 2026, weights released 14 August 2026.

What the comparison is actually deciding

If you are building a desktop application that needs to classify text, follow a structured format, summarise a document or route an operating-system request without a network round trip, the Aion model is in the conversation and Q​wen3.8-27B is not. The 32K window is enough for a document summary and nowhere near enough for a repository-wide coding task. The MIT licence removes the legal review. The ONNX WebGPU build runs where a 55.6 GB checkpoint cannot go.

If you need an agent that works across long contexts, reads screenshots, writes code against a large codebase or runs multi-step tool loops, the reverse holds. Q​wen3.8-27B has the context, the modality and independent measurement, and its weaknesses — latency and token verbosity — are known quantities you can price. The Aion model has no independent run at all, which in practice means you are the one doing the evaluation, on your own tasks, before it touches anything that matters.

There is a third position that is probably the right one for most teams: not choosing. A 2.3B on-device model and a 27B hosted model are complementary, and the architecture that uses both — a small local model for the frequent, cheap, latency-sensitive calls and a large hosted model for the rare, expensive, hard ones — is the one that survives contact with a bill. Q​wen3.8-27B is available through OrcaRouter as qwen/qwen3.8-27b at $0.33 per million input tokens and $2.40 per million output, on the same key and endpoint as the Q​wen3.8-Flash and Q​wen3.8-Max tiers, with automatic failover between providers and no per-token markup over the provider's list price. We run those weights ourselves, so a vendor price change lands here the same day rather than after a repricing cycle. Aion-1.0-Instruct-2.3B is not on OrcaRouter — it is a download you run yourself, and the two can sit in the same application without either one knowing about the other.

Screenshot of the OrcaRouter model page for qwen/qwen3.8-27b, showing the Qwen3.8 27B listing with its 262K-token context, text and image and video input, text output, and pricing.

What to watch, and what would change this piece

Three things would materially change the picture, and none of them has happened yet. Microsoft publishing an announcement, a model card with training details, or a benchmark run outside the company's own harness. Artificial Analysis adding an Aion entry, which would give the newer model the independent counterweight Q​wen already has. Or the quantised BF16-versus-INT4 story resolving into a published accuracy delta, which is the number that decides whether on-device quantisation actually holds up on this architecture.

Until then, treat Aion-1.0-Instruct-2.3B as what the repository shows: a small, permissively licensed, text-only Windows-oriented model, published without ceremony, one day old, with a vendor benchmark table and no independent check on it. Treat Q​wen3.8-27B as a known quantity with published third-party measurement, a context window eight times larger, and a serving bill to match. The wrong question is which one wins. The right one is which of those two objects your application is missing.