Ternary Bonsai 2 27B vs Qwen3.8-27B IQ2_XXS GGUF: Fewer Bits, Better Scores

Ternary Bonsai 2 27B is smaller than the IQ2_XXS GGUF build of Qwen3.8-27B and, on the numbers published with it, also better — which should not be possible if all that separated them were a bit budget. The Bonsai build carries 1.76 bits per weight in a 5.93 GB file. The conventional 2-bit quantization of the same base model carries roughly 2.2 bits per weight in a file around 7.3 GB. Prism ML, which announced the ternary build on 17 September 2026, reports a 20-benchmark average of 83.9 for its model against 75.2 for the IQ2_XXS comparison point — an 8.7-point gap in favour of the model that uses fewer bits.

That inversion is the whole story, and it is not a trick. The two files are produced by different processes at different stages of the model's life, and the difference between those processes is worth more than the difference in bit width. It is also the comparison where the popular advice — "just grab a 2-bit quant, they're fine now" — runs into its clearest counterexample, and where the counterexample has an independent measurement behind it rather than a vendor's word.

The paradox is the mechanism

IQ2_XXS is a post-training quantization format. The model is trained to convergence in full precision and then, afterwards, its weights are rounded into a low-bit representation chosen by a fitting procedure. The model never gets a chance to adapt; it is measured after the fact and approximated. The i-quant family improves on older k-quants by using an importance matrix — a calibration pass that decides which weights deserve more of the available precision — but the fundamental order of operations is unchanged. Train, then compress.

Bonsai inverts that order. Prism ML's description of its own method is that it abandoned post-training quantization entirely and enforced the ternary constraint during training: the forward pass computes with weights restricted to {−1, 0, +1}, while the backward pass still carries full-precision gradients. The model spends its training learning representations that survive the constraint, rather than having the constraint imposed on representations that never expected it. The algorithm is described as proprietary IP, but the shape of it will be familiar to anyone who has read the BitNet line of work.

Two refinements sit on top, and both are visible in the arithmetic. The first is selective precision: 26.2 million parameters — about 0.098% of the model, roughly 52 MB at bf16, mostly the recurrent state path of the linear-attention layers plus normalization weights — are kept in full precision rather than ternarized. The second is a blockwise rotation. Each weight matrix is transformed by a Walsh–Hadamard rotation at block size 1,024 before the ternary values are chosen, which spreads outliers across coordinates and makes a three-level approximation less destructive. The rotation is folded into the stored weights, so it costs no extra bytes; the matching transform is applied to activations at runtime instead.

That last detail has a consequence you meet at the first attempt to run the thing, and it is the real cost of the approach.

Which IQ2_XXS you mean, and what it actually scores

Before comparing quality, a correction that most coverage skips: "IQ2_XXS" is not one artifact. It is a llama.cpp quantization type, and the same type applied by different toolchains to the same base model produces files that differ meaningfully in size and substantially in quality.

The clearest public evidence is an independent comparison posted on 15 August 2026 by a user investigating whether a sub-2-bit Qwen3.8-27B was worth building at all. The test is a wikitext-2 KL-divergence measurement, 100 chunks at 512 context, against a locally built Q8_0 reference with a perplexity of 6.7500, with every candidate using the same importance matrix, corpus, baseline and llama.cpp build in one sitting. The results:

• unsloth UD-IQ2_XXS — 8.39 GiB, perplexity 7.6528, mean KLD 0.146, median KLD 0.076, top-1 agreement 82.98%

• bartowski IQ2_XXS — 8.75 GiB, perplexity 8.5352, mean KLD 0.301, median KLD 0.162, top-1 agreement 76.53%

The dynamic variant is 0.36 GiB smaller than the static one and about 2.1x better on mean KLD — at 1.13x the baseline perplexity. Two files wearing the same label, separated by a factor of two on the metric that measures how far the output distribution has drifted. The same test found every sub-2-bit candidate worse than both published IQ2 options, which is why the practical floor for this base model sits at IQ2 rather than below it.

This matters for the comparison because it changes what "the 7.3 GB IQ2_XXS build" refers to. Prism ML's figure comes from its own quantization of the base model at 2.2 bits per weight. An unsloth dynamic build of the same family measures 8.39 GiB. A bartowski build measures 8.75 GiB. The size gap between Bonsai and "IQ2_XXS" is therefore somewhere between 1.4x and 1.5x depending on which build you mean — larger than the 1.23x the headline implies, and all of it before the quality comparison starts.

Where the post-training build breaks, and why it is easy to miss

The 8.7-point aggregate gap is the least informative way to state the difference, because the degradation in the IQ2_XXS build is not uniform. It is selective, and the pattern is the opposite of what most people would predict.

• MMLU-Redux — the post-training build holds up respectably, in the mid-to-high 80s

• GPQA Diamond — around 65.5, against 85.76 for the ternary build

• AIME26 — in the range of 57.5 to 78.6 depending on the build, against 95.83 for the ternary build

• LiveCodeBench — in the range of 56.4 to 70.05, against 90.07 for the ternary build

The exact IQ2 figures move between Prism ML's suites and the community measurements, and the ranges above span both, but the shape is consistent across every source: surface knowledge survives, and anything requiring a sustained chain of reasoning degrades sharply. That is precisely the failure mode that a casual test will not find. Ask a 2-bit build to summarize a document or answer a factual question and it performs like a much larger model. Ask it to hold a multi-step derivation or produce non-trivial code and the collapse is sudden rather than gradual. This is why "it felt fine when I tried it" is not evidence about a quantized model — it is evidence about the prompts you happened to try.

The ternary build does not show that collapse on the same benchmarks. Prism ML reports AIME26 at 95.83 against 94.58 for its full-precision base, and LiveCodeBench at 90.07 against 90.05 — effectively level, and the single most useful claim in the release because it says the training-time constraint bought the thing the post-training one loses.

The counterargument, which is real and which the quality table does not capture

Everything above favours the ternary build on quality per byte. There is one dimension on which the conventional GGUF wins outright, and it is not a small one: it runs on the software you already have.

The IQ2_XXS build of Qwen3.8-27B is a stock llama.cpp artifact. It loads in llama.cpp, Ollama, LM Studio, Jan, and anything else that links ggml, on every platform ggml supports, with no fork required and no special kernels. Its importance matrix can be rebuilt or replaced. It behaves like every other quantized model you have on disk.

Ternary Bonsai 2 27B does not. The ternary kernels for this architecture's hybrid attention live in Prism ML's own llama.cpp fork. Stock llama.cpp rejects PTQ1_0 and PQ2_0 as unrecognised types — and has no idea what to do with the rotated basis those packs assume. The MLX build for Apple Silicon has Metal and CPU kernels but no CUDA path, so on an NVIDIA machine it falls back to a CPU forward pass that can take minutes. Prism ML does list integration with several runtimes, but the underlying point stands: this model's usability is bounded by whether your runtime of choice has been taught about it.

That is the honest trade. You are choosing between a build that is 1.4x bigger, measurably worse at exactly the tasks you most likely want a 27B model for, and universally runnable — and a build that is smaller and better, on an ecosystem that currently amounts to one lab's fork plus whichever runtimes have adopted it.

What it takes to run either one

The memory arithmetic is closer than the file sizes suggest, because the files are not the only thing in VRAM.

• IQ2_XXS build — 8.39 to 8.75 GiB of weights, leaving roughly 7.5 GB free on a 16 GB card for context and draft models. The independent test above notes specifically that an 8.39 GiB build already accommodates a 2.1 GB draft model alongside it.

• Ternary Bonsai 2 27B — 5.93 GB for the PTQ1_0 language model, plus the 0.63 GB vision tower if you use images at all, plus context. Prism ML's PQ2_0 packing costs 7.25 GB and is the faster option on hardware limited by instruction throughput rather than bandwidth.

On speed, the reported ternary figures are 142.5 tok/s decode on an RTX 5090 and 46.8 tok/s on an Apple M5 Max; third-party reporting on the IQ2 build is thinner, with a Vulkan measurement on dual Radeon cards around 3.8 tok/s generation, though that number says more about that particular backend than about the quantization. Treat throughput figures on both sides as strongly hardware-specific.

Where OrcaRouter fits, and where it does not

Neither of these files is something a router serves. They are local artifacts, and the honest framing is that OrcaRouter does not host either one — this is a comparison about what runs on your own hardware. What does sit on our side is the model they were both derived from. Qwen3.8-27B full precision is available through OrcaRouter's own infrastructure at $0.33 per million input tokens and $2.40 per million output tokens, with the model's native 262K context and its low/medium/high reasoning-effort control intact.

That gives a hybrid setup it is worth being concrete about. Run the compressed build locally for the tasks it is good at, and route the occasional request that needs full precision to the hosted base model through the same key — no second vendor contract, no code change, because both sides speak the same API. If the local build turns out to be wrong for a workload, the request that fails can be failed over automatically rather than returned as an error, which is a cheaper way to discover that a quantization is unsuitable than discovering it in production.

The short version

• On published quality per byte, the ternary build wins clearly, and the win is concentrated in reasoning and code — the categories where the post-training build degrades most.

• On portability, the IQ2_XXS build wins just as clearly. Stock runtimes everywhere, no fork, no special kernels, no silent-garbage failure mode.

• The comparison is not "1.76 bits versus 2.2 bits". It is "a representation chosen during training versus one fitted afterwards", and the 0.44-bit difference is a rounding error next to that.

• Every quality figure for the ternary build in this article is Prism ML's own measurement on a suite Prism ML chose. The KLD results for the IQ2 builds are independent, single-run, and specific to one base model and one test set; they are the strongest third-party evidence in this comparison and they say nothing about Bonsai.

The gap will close from the other direction too, and probably soon. If the ternary kernels land upstream in llama.cpp, the one serious objection to Bonsai evaporates and the choice becomes straightforward. Until then the IQ2_XXS build keeps a real advantage that has nothing to do with how good it is.

If you are deciding this week, the test that settles it takes an afternoon: take twenty prompts from your own workload that require more than one reasoning step, run both builds, and score the answers blind. The published tables tell you where to look. They cannot tell you whether your work lives there.