Ternary Bonsai 2 27B vs Bonsai 27B: Two Months, Two Base Models, One Missing Variant

Ternary Bonsai 2 27B arrived on 17 September 2026, two months after Bonsai 27B landed on 14 July 2026, and the headline comparison between them is a single pair of numbers: the first generation kept about 95% of its full-precision base model's benchmark average, and the second keeps 98.2%. That reads like a straightforward generational improvement, and it mostly is — but the two figures are not measuring the same thing, because the base model changed underneath them. The July release compressed Qwen3.6-27B. The September release compresses Qwen3.8-27B. Part of the quality gain belongs to the compression recipe and part of it belongs to the newer Qwen3.8-27B, and no published figure separates the two.

There is a second difference between the generations that has had far less attention and matters more to a specific group of users: the first Bonsai shipped in two variants and the second ships in one. The 3.9 GB build that made a 27B-class model fit an iPhone 17 Pro has no successor in this release. If that footprint is the reason you were interested in Bonsai at all, the newer generation is not an upgrade — it is a different product that does not cover your case.

What the first generation actually shipped

Bonsai 27B was released on 14 July 2026 under Apache 2.0 as two artifacts built on the same base model, and the split between them was the point of the release.

• Ternary Bonsai 27B — ternary {−1, 0, +1} weights with FP16 group-wise scaling, 1.71 effective bits per weight, a 5.9 GB footprint. The quality-oriented build, positioned for an everyday laptop with full reasoning, tool-calling and agentic capability.

• 1-bit Bonsai 27B — binary {−1, +1} weights with the same group-wise scaling, 1.125 effective bits per weight, a 3.9 GB footprint. The footprint-oriented build, sized to fit an iPhone 17 Pro's memory budget.

Both carried a 262K-token context, both kept a compact 4-bit vision tower so the model stayed multimodal, and both supported speculative decoding with a DSpark drafter. Prism ML's framing at the time was that the low-bit representation ran end to end — embeddings, attention, MLPs and the LM head — with no higher-precision escape hatches, and that the 1-bit build was the first 27B-class model to run on a phone at all. Reported throughput on the 1-bit variant was around 11 tokens per second on an iPhone 17 Pro, 87 tok/s on an Apple M5 Max, and 163 tok/s on an RTX 5090; the ternary variant was quoted at 58 tok/s on the M5 Max and 134 tok/s on the 5090.

The quality cost was reported alongside those numbers rather than buried. On a 15-benchmark thinking-mode suite the full-precision base scored 85.0, the ternary build 80.5 — about 95% — and the 1-bit build 76.1, about 90%. The degradation was concentrated in agentic tool calling, which fell from 80.0 to 66.0 on the 1-bit build, and in vision, which fell from 72.6 to 59.6. Math and coding held up considerably better in both variants.

What the second generation changed

Ternary Bonsai 2 27B keeps the recipe and changes the inputs. The ternary representation is still {−1, 0, +1} with one FP16 scale per group of 128 weights, now packing to 1.76 bits per weight in a 5.93 GB file, with 262K context and the same separate vision tower — 0.63 GB at 4-bit in this release, loaded only when an image arrives.

Two things are genuinely new, and both are described as the reason the retention figure moved.

The first is selective precision. Unlike the July build, which ternarized essentially everything, Bonsai 2 holds 26,238,464 parameters in full precision — 0.0976% of the language model, about 52 MB at bf16, concentrated in the recurrent state path of the linear-attention layers plus normalization weights. That is a small concession in bytes and apparently a large one in behaviour.

The second is a rotated weight basis. Weight matrices are stored after a blockwise Walsh–Hadamard rotation at block size 1,024, with the matching transform applied to activations at runtime, on the theory that spreading outliers across coordinates makes a three-level approximation less lossy. It costs no extra storage because the rotation is folded into the weights, but it does sit on the computational path.

Then there is the change that is not a technique at all: the base model. Qwen3.8-27B is a hybrid-attention design — roughly 75% linear attention, 25% full attention — where its predecessor was not. Prism ML's reported category scores show what that move bought. Instruction following sits at 82.66 for Bonsai 2, against 74.53 for Qwen3.6-27B, the base the first generation compressed. Reasoning and knowledge come in at 83.95 against 84.71 for the older base, and coding at 81.58 against 82.57. The new base is better at instruction following by a wide margin and slightly behind on two other categories, which is exactly the kind of profile that makes cross-generation retention percentages unhelpful on their own.

Why the two retention figures are not comparable

95% and 98.2% look like two readings on one scale. They are not, for three reasons that are worth keeping straight before concluding that the recipe improved by 3.2 points.

• The denominators differ. The first generation's 95% was measured on a 15-benchmark suite against Qwen3.6-27B. The second's 98.2% comes from a 20-benchmark suite against Qwen3.8-27B. Different suites, different baselines, different difficulty mixes.

• The baselines moved independently. Some of the gain in retention is the compressed model getting better at compressing, and some is the base model changing in ways that happen to be friendlier to ternary weights. Nothing published separates those contributions.

• Retention is relative, so it can rise while absolute capability falls in a category. A model that retains 99% of a weaker parent can still trail a model that retains 96% of a stronger one.

The absolute comparison is more informative than the relative one, and on that footing the story is cleaner. Bonsai 2's aggregate of 83.9 is above the 83.6 that full-precision Qwen3.6-27B scored on the older suite — meaning the compressed successor is now ahead of the uncompressed model it replaced a generation earlier. The first-generation ternary build scored 80.5 on its own suite. Both of those figures are Prism ML's, and the suites differ, so read the ordering rather than the decimals.

The variant that did not come back

This is the part of the comparison that changes a buying decision rather than a benchmark chart.

There is no 1-bit Bonsai 2. The September release ships a ternary build, in two packings — PTQ1_0 at 1.76 bits per weight and 5.93 GB, and PQ2_0 at 2.16 bits per weight and 7.25 GB — plus an MLX container for Apple Silicon. There is no 3.9 GB binary variant, and no announcement of one. The 3.9 GB phone-class figure that appears in current coverage still refers to the July model.

The practical consequence is direct. If your target is an iPhone or iPad, or any device where a 5.9 GB language model plus a 0.63 GB vision tower plus a context budget does not fit, then the first-generation 1-bit build remains the only option in this family, and it will remain so until a 1-bit Bonsai 2 exists. Upgrading the ternary path does not upgrade that path. Anyone who reads "Bonsai 2 is better" and re-downloads onto a phone will find the file does not fit.

If you are on a laptop or desktop, the calculus is the opposite: there is no reason to run the July ternary build when the September one is smaller per unit of quality, better on the benchmarks that matter, and carries the same 262K context.

Speed, where the generations are genuinely hard to rank

Throughput is the part of this comparison where the honest answer is that the published numbers do not support a clean ranking, and it is worth saying so rather than picking the flattering pair.

The second generation's standardised measurements at batch size 1 with the vision tower excluded are 142.5 tok/s decode on an RTX 5090 on the PQ2_0 packing, 46.8 tok/s on an Apple M5 Max, 27.7 on an M5 Pro, and 18.0 on an M4 Pro. The first generation quoted 134 tok/s on an RTX 5090 and 58 tok/s on an M5 Max for its ternary build. The 5090 figure moves modestly in the expected direction. The M5 Max figure moves the other way — 58 down to 46.8 — which is not what a two-month generational step is supposed to look like.

Two caveats stop that from being a finding. The measurement bases differ between releases, and at least one published M5 Max figure for the newer model has been attributed to a build predating the rotation optimisation. But it is worth flagging as an open question, because the mechanism that would explain it is sitting in the release notes: the rotated basis puts a transform on the critical path of every projection at batch size 1, and Apple Silicon decode is the regime where that hurts most. The technique that buys quality may cost decode throughput, and on unified-memory hardware that trade is at its sharpest.

The MLX container adds a separate wrinkle for Apple users. It is an affine 2-bit format whose block stores both a scale and a bias for every group of 128 weights, but ternary weights need only the scale, so the bias is dead weight — the block costs 36 bytes per 128 weights instead of 34, and the packed rate lands at 2.25 bits per weight with a measured file size of 8.005 GiB. It is a different container carrying the same values, and it is the pack the demonstration setup downloads by default.

Running either generation

Both generations share one operational constraint that no version of this model has escaped: neither one runs on stock llama.cpp. The ternary kernels for this architecture live in Prism ML's own fork, stock llama.cpp rejects the current packings as unknown, and — worse — it will load the older ternary format without complaint and produce fluent garbage, because it does not apply the rotation the weights assume. The MLX build carries Metal and CPU kernels but no CUDA path. Whichever generation you pick, the runtime question is answered by Prism ML's own distribution or by a runtime that has adopted its kernels, not by the ggml ecosystem at large.

Where OrcaRouter fits into a decision like this is one level up. Neither Bonsai generation is hosted here — they are downloads you run on your own hardware. What the routing layer is useful for is the boundary: the requests your local model should not be answering. Define the escalation policy once in the routing configuration rather than in application code, so that a locally served tier passes long-context, vision-heavy or otherwise out-of-scope requests up to a hosted model instead of failing them, and so the fallback survives whichever generation of Bonsai you happen to have installed. Both the local tier and the hosted one then sit behind a single key, and the policy lives in one place when the next Bonsai generation lands and the tier boundaries move again.

What to do with this

• If you run the July ternary build on a laptop or desktop — move to Ternary Bonsai 2 27B. It is a better model at roughly the same footprint, and the category scores where it wins are the ones that matter for agentic and instruction-following work.

• If you run the July 1-bit build on a phone — stay. There is no successor, and the 5.93 GB ternary build is not a drop-in replacement for a 3.9 GB one.

• If you are evaluating the family for the first time — decide the footprint first, then the generation. The variant you need determines which release you are shopping in, and that ordering is the reverse of how this upgrade is usually described.

• If you are choosing on benchmarks — treat 95% and 98.2% as two different measurements rather than two points on a line, and treat every figure in both as the vendor's own until an independent evaluation of the September model appears. That evaluation is the thing to watch, because it is the first one that will be able to compare the two generations on a common basis.