Bonsai 2 vs Bonsai 27B: One of Them You Can Pull, the Other You Cannot

The most concrete difference between Bonsai 2 and Bonsai 27B has nothing to do with benchmarks. On 17 October 2026 an Ollama library entry went live covering the whole Bonsai 2 family — Ternary Bonsai 2 2B, Ternary Bonsai 2 4B, Ternary Bonsai 2 9B and Bonsai 2 27B — and there is no equivalent entry for the first generation. Check for bonsai-27b, ternary-bonsai-27b or any variant name on that registry and you get a 404. Bonsai 27B, released by Prism ML on 14 July 2026, has to be downloaded from the vendor's Hugging Face repositories and run against the Prism ML llama.cpp fork. Bonsai 2 has a one-line install.

That asymmetry is the whole practical shape of the choice, and it is easy to miss because the two names look like one product line with a version bump. They are two generations, two base models, two different sets of packings, and — as of this week — two different delivery mechanisms. What follows is what actually changed between them, where the older one still wins, and the one configuration where the answer is neither.

What each one is

Both are ternary compressions of a Q​wen hybrid-attention model. Neither is a newly trained model — the architecture, the weights and the quantisation-aware training come from the Q​wen base, and what Prism ML contributes is the ternary representation and the kernels that consume it.

• Generation — Bonsai 2 (September 2026, extended October 2026) vs Bonsai 27B (14 July 2026)

• Base model at the 27B size — Qwen3.8-27B vs Qwen3.6-27B

• Base models across the family — Qwen3.5-2B, -4B, -9B and Qwen3.8-27B under Bonsai 2; a single Qwen3.6-27B under the first generation

• Sizes offered — four (2B, 4B, 9B, 27B) vs one (27B)

• Two operating points at 27B — no, one: Bonsai 2 27B ships ternary only, while Bonsai 27B shipped both a ternary build and a binary one

• Smallest 27B-class footprint — 5.95 GB PTQ1_0 in Bonsai 2 vs 3.9 GB in the first generation's 1-bit build

• Context — 262,144 tokens inherited from the Q​wen backbone on both

• Licence — Apache 2.0 on both

• Delivery — Ollama tags and Hugging Face for Bonsai 2; Hugging Face plus the Prism ML llama.cpp fork for Bonsai 27B

The row that decides more deployments than the rest is the fifth one. Bonsai 27B shipped two ways to spend memory: a ternary build at roughly 7.2 GB and a binary build at 3.9 GB. Bonsai 2 27B ships one — a ternary build at 5.95 GB in the dense-trit packing or 7.21 GB in the 2-bit-slot packing — and there is no binary companion in this generation. If you were running the first generation's 3.9 GB build and expecting a like-for-like successor, there is not one at 27B.

What the generation actually bought

Prism ML's own numbers, on Prism ML's own harness, with no independent reproduction available: the first-generation ternary Bonsai 27B reported a thinking-mode average of 80.49 against 85.07 for its Qwen3.6-27B FP16 reference — 94.6% retained across 15 benchmarks. Bonsai 2 27B reports 84.78 against 86.32 for Qwen3.8-27B FP16 — 98.2% across 14 benchmarks. In absolute terms the compressed model gained about 4.3 points on the vendor's scale.

Two caveats belong in the same breath, because the headline invites a cleaner reading than the data supports. First, the benchmark suites are not the same set — 15 for one, 14 for the other — and the baseline moved from Qwen3.6-27B to Qwen3.8-27B, which was itself a stronger model. The vendor's own 27B model card makes the second point explicit by recomputing the previous release on the new 14-benchmark suite for a like-for-like row: on that common footing the previous release scores 80.98 and Bonsai 2 27B scores 84.78. So the honest reading of the change is roughly 3.8 points against the same benchmark set, with part of that coming from the base model and part from the ternary training. The retention percentage rose from 94.6% to 98.2% partly because the denominator moved.

Second, the shape of the loss is more informative than the aggregate. On the same 14-benchmark suite, Bonsai 2 27B gives up almost nothing where sustained reasoning shows up: math 96.57 against 97.06, coding 89.42 against 89.07 — level, marginally ahead — and instruction following 82.66 against 81.25, ahead. Knowledge and reasoning falls 85.55 to 79.86, and vision 71.36 to 66.19. That is the profile you would expect from a 1.72-bits-per-weight representation: the reasoning core survives, the factual recall and image understanding take the hit. If your workload is code and mathematics, the compression is close to free. If it is retrieval-shaped question answering over a corpus, it is not.

The 3.9 GB slot is where this gets interesting

Here is the part the two-generation framing hides. The first generation's most distinctive artefact was the binary build: a 27B model in 3.9 GB, retaining a vendor-reported 89.5% of FP16 on a 15-benchmark average of 76.11. Bonsai 2 has no binary 27B. But Bonsai 2 does have a 9B at 1.997 GB in the dense-trit packing and 2.42 GB in the 2-bit-slot packing — roughly half the footprint of the binary 27B, from the newer generation and the newer base model.

Is the 9B better than the 1-bit 27B? The vendor does not report a number that answers that, and you should not construct one. The two models were evaluated on different suites — 19 benchmarks across six skill areas for the small Bonsai 2 models, 15 for the first-generation 27B — against different baselines, and the small models' release reports category retention against their own full-precision Qwen3.5 counterparts rather than a single absolute average comparable to the 27B's 76.11. The comparisons that do exist are per-benchmark and point in both directions. On the vendor's numbers the 9B scores 71.81 on BFCLv3 and 73.02 on Tau2Bench, while the binary 27B posted 74.41 on BFCL v3 — close enough that the suite difference swallows it.

What can be said without overreach is narrower and still useful: the footprint class the binary 27B used to own alone now has a cheaper entrant from the next generation, and the two are close enough that nobody should pick between them from a vendor table. That is a case for measuring on your own prompts, not for trusting a number. It is also the one place in this comparison where the answer is genuinely neither of the two models in the title.

Running them: the numbers and the catch

Throughput figures are vendor-measured and were not taken on a common machine, so read them as orders of magnitude rather than a ranking.

• Bonsai 2 27B on an Apple M5 Max — about 47 tok/s on the earlier pre-rotation build, pending re-measurement on the current stack

• Bonsai 2 27B on an M5 Pro — 28.1 tok/s in the 2-bit-slot packing, at roughly 27.5 W on the GPU rail

• Bonsai 2 27B on an RTX 5090 — 129.9 tok/s in the 2-bit-slot packing, 120.5 in the dense-trit packing

• First-generation ternary Bonsai 27B on an M5 Max — 44.0 tok/s; on an H100, 98.0 tok/s, rising to about 131.8 with its DSpark drafter

• First-generation binary Bonsai 27B on an iPhone 17 Pro Max — about 11 tok/s, thermally limited

The catch is packaging, and it cuts the same way for both generations. Neither generation's ternary kernels are in upstream llama.cpp. The first generation needs the Prism ML fork for every file. Bonsai 2 ships three packings of the same weights and upstream llama.cpp runs exactly one of them — Q2_0, in recent master — while rejecting the smaller PTQ1_0 and the faster PQ2_0 as unknown types. So the dense-trit pack that lands closest to the theoretical 1.72 bits per weight is also the one that requires a forked build.

That is worth sitting with, because it explains the Ollama listing. The Ollama tags for Bonsai 2 serve Q2_0: the 9B tag's model layer measures 2.557 GB, byte-for-byte the Q2_0 file size on Hugging Face, and the smallest PTQ1_0 pack is absent from the registry entirely. Whatever else that listing is, it is not a route to the smallest artefact. Convenience and footprint are being traded here, and the trade is invisible unless you read the file sizes.

Getting to either one

If you are on Ollama, the decision is made for you: Bonsai 2, four sizes, ollama pull bonsai-2:27b or :9b, roughly 8.3 GB and 3.2 GB on disk including the vision projector. If you need the dense-trit packing, the speculative-decoding drafters, or anything from the first generation, it is a Hugging Face download and a fork build, and the vendor's demo repository is the tested reference for that path.

Where a routing layer genuinely helps here is on the other side of the question. You cannot route to either of these — they are local weights, and OrcaRouter does not serve any Bonsai size today. What OrcaRouter gives you is the comparison baseline: one API in front of 200-plus hosted models at 0% markup, provider list price passed through, with automatic failover between providers. The sensible sequence for most teams is to establish on a hosted model what a task actually needs, then decide whether a 2 GB, a 3.9 GB or a 6 GB local model clears that bar — rather than guessing from a retention percentage.

Which one, concretely

• You want the newest ternary generation at 27B and a one-command install — Bonsai 2 27B through Ollama, accepting the Q2_0 packing and the larger on-disk size.

• You want the smallest possible 27B-class artefacts, or you rely on the DSpark drafter — first-generation Bonsai 27B from Hugging Face, with the fork build. Nothing in Bonsai 2 replaces it at 27B.

• You were running the 3.9 GB binary build and need that footprint to hold — do not assume Bonsai 2 has a successor. Either stay, or measure Bonsai 2 9B against your own prompts; it is half the size and a generation newer, and the vendor has not published a comparison that settles it.

• You need four size classes from one runtime — Bonsai 2 is the only generation that offers that.

One thing to watch that would simplify all of this: a ptq1_0 tag appearing on the Ollama listing. That would mean the dense-trit kernels reached the engine Ollama ships, the fork requirement drops away for the smallest pack, and the convenience-versus-footprint trade stops being a trade.

None of that is hypothetical: the documented AI failures and agent incidents behind releases like this one are collected in the OrcaRouter AI Incident Archive, where each entry carries its own sources, severity, confidence grade and disputed flag.