Bonsai 27B: The 27B Model in the Bonsai Family, Explained

Bonsai 27B is the 27-billion-parameter model in PrismML's Bonsai family, and it is the one people mean when they type "bonsai 27b" — but the name on its own will not get you there, because three different things answer to something close to it. Bonsai 27B is a low-bit rebuild of Qwen3.6-27B, dated 14 July 2026 by the vendor's own announcement, and it ships as two packs of the same model: 1-bit Bonsai 27B, which stores binary {−1, +1} weights in about 3.9 GB, and Ternary Bonsai 27B, which stores ternary {−1, 0, +1} weights at 1.71 effective bits per weight — 5.9 GB in the vendor's headline figure, 7.165 GB as the pack actually ships. It is not Ternary Bonsai 2 27B, the separately named second generation built on Qwen3.8-27B, and it is not the 2-billion-parameter vision-language model that PrismML built for Snapdragon AR1 Gen 1 smart glasses. Both packs of Bonsai 27B carry a 262,000-token context, a compact 4-bit vision tower, and an Apache 2.0 licence. The model is Bonsai 27B; the decision that follows is which of its two packs fits your memory budget.

This page is the reference for that model. It is not an announcement: Bonsai 27B has been available since July 2026, and the questions worth answering about it now are the ones a reader arrives with — what exactly it is, what the name distinguishes, and what running it actually costs.

Which one is the 27B

The confusability here is real, and it is worth resolving before anything else, because the wrong guess sends you to a 2-billion-parameter model or to the wrong generation of the right one.

• Bonsai — the family name, and the vendor's name for it. On its own it identifies nothing you can download. PrismML has shipped Bonsai 1.7B, Bonsai 4B, Bonsai 8B, Bonsai 27B and Bonsai Image, plus the 2-billion-parameter glasses model, and an earlier April 2026 release called Ternary Bonsai for the smaller line. A search for the bare word lands in that set at random.

• Bonsai 27B — the subject of this page. Generation one of the 27B line, derived from Qwen3.6-27B, dated 14 July 2026. When a page says "Bonsai 27B" with no other qualifier and no "2", this is the model it means, and this is the name to use.

• Ternary Bonsai 27B — not a different model. It is the quality-oriented pack of Bonsai 27B: the same 27B model, the same architecture and the same base, with ternary weights instead of binary ones. The repository is named Ternary-Bonsai-27B-gguf, and its model card names Qwen3.6-27B as the base exactly as the binary card does. "Ternary" here is a weight format, not a version.

• Ternary Bonsai 2 27B — a genuinely different model. The second generation of the 27B line, built on Qwen3.8-27B and announced 17 September 2026, with a different parameter split and a different quality claim. The "2" carries the whole distinction.

• 1-bit Bonsai 2B — a different family member at a different scale: a 2-billion-parameter vision-language model for glasses-class hardware, not a 27B model at any precision.

So if you came here to settle which of these is the 27B model, the answer is the third and fourth lines above are the same model at two weight formats, and the fifth is the one that is not. Practically: call it Bonsai 27B, and add the pack when it matters.

The specification, from the model card

Every figure in this section comes from PrismML's published model cards for the two packs, and it is the vendor's own statement about its own model. Where a number is a claim rather than a measurement, it is labelled as one.

• Parameters — about 27.3B in total, and the "27B" refers to that whole. The split is worth knowing because it explains the memory behaviour: roughly 24.8B sits in the language backbone across 64 blocks, roughly 2.5B in the embedding and LM head, and about 0.46B in a 27-block vision tower. The low-bit representation is applied end to end — embeddings, attention, MLPs and the LM head — with no higher-precision escape hatch anywhere in the language network.

• Context — a full 262,000 tokens, with no long-context variant and no reduced-context build. The cost of that window is managed rather than paid outright: the backbone is hybrid, so only 16 of its 64 layers keep a full attention cache, and that cache is stored in a near-lossless 4-bit form. At the full 262K window the compressed cache is what keeps the model inside roughly 9.4 GB on the binary pack and 12.8 GB on the ternary one.

• Modality — text and images, both packs. The vision tower ships in a compact 4-bit form (0.63 GB) with a BF16 copy (0.93 GB) alongside it. Vision is also where the compression costs the most, which is covered below.

• Weight format and size — 1-bit Bonsai 27B uses binary {−1, +1} weights with FP16 group-wise scaling, packed as GGUF Q1_0_g128 at a true 1.125 effective bits per weight; the shipped file is 3.803 GB, and the card rounds the deployed footprint to about 3.9 GB. Ternary Bonsai 27B uses ternary {−1, 0, +1} weights with the same group-wise scaling; a ternary value carries log₂3 ≈ 1.585 bits of information, so the representation's floor is ~1.71 bits per weight, which comes to 5.9 GB ideal. The published ternary pack is larger than that ideal, because today's kernels store each ternary value in a 2-bit slot — 2.125 bits per weight deployed — and the shipped file is 7.165 GB. PrismML states both numbers rather than only the flattering one, and the gap between them is a kernel limitation rather than a modelling choice.

• Licence — Apache 2.0 on both packs, stated on both repositories and in the vendor's announcement. That is a permissive licence with no field-of-use restriction, which matters more at this weight size than at most.

• Base model — Qwen3.6-27B in both cases. This is a re-compression of somebody else's model rather than a model trained from scratch, and the consequence is that Bonsai 27B's capability ceiling and its refresh cadence both track the upstream 27B line rather than PrismML's.

The ternary relationship, stated once

The two packs are one model, and the honest summary is that ternary buys quality and binary buys footprint, with a gap between them that is smaller than the 3.9 GB against 7.2 GB size difference suggests.

On PrismML's own 15-benchmark thinking-mode suite, the FP16 Qwen3.6-27B baseline scores 85.07. Ternary Bonsai 27B scores 80.49 — 94.6% retention, which the vendor rounds to about 95%. 1-bit Bonsai 27B scores 76.11 — 89.5%, rounded to about 90%. So the quality-oriented pack recovers roughly half the distance between the binary pack and full precision while costing about 3.3 GB more, and the binary pack is the one that fits a 12 GB phone. The same suite shows a 4-bit Q4_K_XL build of the same baseline at 84.99, or 99.9% retention, in 17.6 GB — which is the comparison that explains why either low-bit pack exists at all, since it beats them on quality and loses on every device that has less than 18 GB to give a model.

Where the binary pack gives ground is uneven, and the category split is the useful part. Against the FP16 baseline it drops from 95.33 to 91.66 on math and 88.74 to 81.88 on coding, but from 78.47 to 65.74 on instruction following, 80.00 to 66.03 on agentic and tool-calling work, and 72.61 to 59.57 on vision. If your workload is arithmetic or code completion, 3.9 GB is close to free. If it is tool-calling agents or image understanding, that is where the compression is actually being paid for.

The model card also publishes a derived figure it calls intelligence density — aggregate score per gigabyte, normalised so that a perfect score approaches infinity. On that measure 1-bit Bonsai 27B reaches 0.530 and Ternary Bonsai 27B 0.400, against 0.199 for a 2-bit IQ2_XXS build of the same baseline, 0.155 for a 4-bit Q4_K_XL build, and 0.051 for the FP16 model. The interesting number in that list is the first comparison: the binary pack beats the conventional 2-bit build on aggregate quality at less than half the footprint, which is the entire thesis of the family in one line.

The full head-to-head between the two packs, and where each one's numbers come from, is set out in the Bonsai vs Bonsai 27B page on this blog, alongside the 2-billion-parameter model that shares the family name.

One question that arrives with this model deserves a factual answer and no more. The packs are low-bit encodings of the same underlying model rather than a re-trained variant, and the repositories publish no separately tuned or uncensored checkpoint next to them — the only full-precision artifact on offer is an unpacked FP16 conversion of the same weights, published so you can compare a compressed pack against its own baseline in stock tooling.

Weights and licence, from the repositories

Bonsai 27B's weights are published on Hugging Face under the prism-ml organisation, and the file listing is the most concrete thing about the release. The binary pack's repository carries the model itself at 3.803 GB, an FP16 GGUF of the same model at 53.808 GB for comparison, a DSpark speculative-decoding drafter at 1.787 GB in 4-bit and 7.292 GB in BF16, and vision towers at 0.931 GB and 0.629 GB. The ternary repository carries its own 7.165 GB pack plus a second packing at 7.585 GB, which is group-64 rather than group-128 and upstream-compatible rather than optimal, and the same drafter and vision files. Both repositories are Apache 2.0 and both name Qwen3.6-27B as the base.

Two things follow from that listing. The first is that 53.808 GB against 3.803 GB is the 14.2x ratio the model card leads with, and it is checkable rather than asserted — the full-precision file is right there. The second is that the release is heavier than the model: if you take the drafter and the vision tower as well, which the multimodal path requires, you are adding roughly 2.4 GB on top of a 3.9 GB model. The 3.9 GB headline is the language weights and not the deployment.

Downloads are the one place the quality-oriented pack leads: as of October 2026 the ternary GGUF repository has passed 590,000 downloads against just over 400,000 for the binary pack, while the MLX containers for both sit above a million each. The shape of that suggests most of this model is being run on Apple hardware rather than on servers, and that the people downloading it are buying the extra five points of retention rather than the smallest possible file — which is roughly what the vendor designed the pair for.

Running it

What you can obtain today is four kinds of artifact per pack, and which one you want is decided by your runtime rather than by quality.

• GGUF for llama.cpp — the primary path, and the one the low-bit kernels were written for. Both packs are llama.cpp builds for CUDA, Metal and CPU, but the 1-bit and ternary kernels live in PrismML's own fork rather than upstream, so stock llama.cpp will not load them correctly yet. The group-64 ternary packing exists precisely for the cases where you need upstream compatibility and are willing to take 7.585 GB instead of 7.165 GB to get it.

• MLX for Apple silicon — separate shipped containers at 5.129 GB for the binary pack and 8.491 GB for the ternary one. Note that the MLX binary container is larger than the GGUF binary pack, which is a container-overhead difference rather than a different model.

• AWQ 4-bit for server frameworks — a 4-bit build of each model for the serving frameworks that expect an AWQ checkpoint, in nine shards totalling 18.70 GB, including a multi-GPU recipe. This is not a low-bit artifact; it is the same model at a precision that makes sense on a rack, and it trades away the entire reason the model exists.

• Unpacked FP16 safetensors — the full ~54 GB model in stock Hugging Face format, published explicitly for frameworks that do not yet support 1-bit weights natively. The vendor says this artifact will be retired once the low-bit kernels land upstream.

Measured decode throughput on the binary pack, from the model card's own table, is 66.4 tokens per second on an M5 Max, 44.2 on an M5 Pro, 26.0 on an M4 Pro and 104.8 on an H100, with prefill at 874, 421, 133 and 2,755 tokens per second respectively. The announcement page quotes higher peaks — up to 87 tokens per second on an M5 Max and 163 on an RTX 5090 — which are the vendor's own best-case configuration rather than the table's. Treat the two sets as different measurement conditions, not as a contradiction, and treat both as vendor-reported. Peak memory, again from the card, is 5.2 GB at a 4K context, 5.6 GB at 10K and 11.6 GB at 100K on the binary pack, and 8.4, 8.7 and 14.7 GB on the ternary pack — with the caveat that those figures predate the 4-bit KV cache, which is what brings the full 262K window down to single-digit gigabytes.

The DSpark drafter is optional and worth understanding before you enable it, because it changes nothing about outputs: it is a six-layer block-parallel transformer conditioned on hidden states tapped from five evenly spaced target layers, it adds about 0.5 GB at serving precision, and it verifies against the target model rather than sampling on its own. That is why the vendor can quote a 1.37x speedup on CUDA without touching the quality figures. It is not enabled by default on Apple silicon.

Where the local model runs out of road is the interesting engineering question, and it is a routing question rather than a model question. A 3.9 GB model on a laptop handles what fits in its context; everything else has to go somewhere, and that somewhere is a hosted model behind an API. OrcaRouter is that layer — one endpoint across 200-plus hosted models with 0% markup on provider list prices, automatic failover across providers, and the escalation rule from a local tier to a hosted one expressed as configuration instead of assembled by hand. It is the layer above a local model, not a replacement for it.

The comparison index

Bonsai 27B sits in a family where each neighbour answers a different question, and these are the pages that take each one — none of them is re-argued here.

• Bonsai vs Bonsai 27B — the 2-billion-parameter glasses model against this one: sixteen times the parameters and two hundred and fifty-six times the context, and why the family name carries no information about what you are downloading. The page is a blog post here, not a hosted model, which is a distinction worth keeping straight in this family.

• Bonsai 27B against Qwen3.8-27B — the head-to-head between this model and the newer base it does not use. A dedicated page in this batch carries that comparison, and because the two models answer to similar names and sit one generation apart, it is worth reading on its own terms rather than as an extension of this specification.

• Bonsai vs Ternary Bonsai 2 27B — the generational jump, and the fact that the two releases report their quality against two different rulers: retention of a baseline in one case, a comparative claim in the other.

• Ternary Bonsai 2 27B — the second generation on its own, on Qwen3.8-27B, for readers who have decided they want the newer model rather than this one.

What is not settled

Every quality figure on this page is PrismML's, measured on PrismML's own 15-benchmark suite, and nobody outside the company has published a reproduction of it. That is the single biggest gap in what is known about Bonsai 27B: the compression claims are unusually specific and unusually checkable, and they remain unchecked by a third party. The vendor is also explicit that agentic coding — long-horizon, multi-file, run-and-repair workflows — is not a strong target of this release and that a variant tuned for it is the next thing on the roadmap, which is consistent with the tool-calling and instruction-following losses in the category split above. Until an independent evaluation exists, both the retention figure and the density ranking should be read as the vendor's position rather than as an established result.

What is settled is smaller and more useful: the weights exist, the sizes are checkable against the file listing, the licence is Apache 2.0, and the two packs differ by about 3.3 GB in exchange for roughly five points of retained quality. That is enough to choose between them. Which of them is right for you depends on whether your workload looks more like arithmetic or more like tool-calling, and the category numbers above are the best evidence available for that decision today.