Sensitivity-Aware Quantization Explained: How OrcaSAQ Decides Which Weights Get More Bits

Sensitivity-aware quantization is the practice of spending your bit budget where it matters: the tensors that quantization hurts most get more bits, and everything else stays at a lower base width. This article explains how OrcaSAQ — our calibration-free, architecture-aware mixed-precision method, released with the GLM-5.3-Flash quant family orcarouter/GLM-5.3-Flash-MLX — decides which of the 37,338 tensors in a 320-billion-parameter Mixture-of-Experts model deserve extra bits, entirely without a calibration dataset. The transferable lesson for anyone quantizing a different MoE: shared experts and down-projections earn the extra bits, and you can find them using nothing more than the upstream release's own quantization metadata.

The short answer

Sensitivity-aware quantization is mixed precision with a policy: each tensor's bit width follows how sensitive it is to quantization error, instead of one width for the whole model. The research literature measures sensitivity with Hessians, Fisher information, or divergence between original and quantized layer outputs, then allocates bits to the layers that hurt most. OrcaSAQ belongs to a smaller family that skips the measurement entirely. It encodes the sensitivity ranking in the architecture itself: it uses architectural and tensor-role priors to decide which weights are fragile, then quantizes everything else at a target base precision.

What that buys you is a fast, deterministic, zero-calibration pipeline. There is no calibration corpus to assemble, no per-layer sensitivity search, and no per-model re-tuning, so the same recipe transfers to a new architecture the day its weights drop. What it gives up is adaptation: a calibration-driven method like GPTQ or AWQ looks at the actual activation distribution of your model and your data, and it will usually squeeze more quality out of the same average bit width. OrcaSAQ's bet is that for Mixture-of-Experts models, the role of a tensor tells you most of what a calibration run would, at a fraction of the cost.

Two ways to find the sensitive weights

Before the policy, the question: how do you know which tensors quantization hurts? The two answers are the whole design space.

• Calibration-driven. Run a small corpus through the model, measure the error each tensor or block causes, and allocate bits to minimize total reconstruction error. GPTQ uses a Hessian-based approximation to the per-layer quantization error; AWQ uses activation statistics to identify the salient weights to protect. The strength is adaptation to your actual data; the costs are a curated corpus, forward passes and inverse-Hessian solves for every layer, and results that shift when the calibration set changes.

• Calibration-free. Decide the sensitivity ranking before seeing any data, from the architecture. In a MoE, you already know the load-bearing roles: the expert that fires on every token, and the projection that writes into the residual stream. Encode that ranking as a fixed policy and apply it mechanically.

OrcaSAQ is firmly in the second camp, and this article is a defense of that camp for MoE quantization — with a clear-eyed account of what you give up for it.

The policy: which tensors get more bits

OrcaSAQ's bit-allocation policy is stated in the model card for orcarouter/GLM-5.3-Flash-MLX, and it reduces to three rules plus one carve-out. The base precision is the build you are making — 6, 4, 3, or 2 bits — and the policy raises specific tensor roles above it:

• Shared expert: base +2 bits. The shared expert fires on every token, so its quantization error is replayed into every single output the model produces. It is the highest-leverage tensor in the model, and it gets the most bits.

• down_proj: base +1 bit. In a SwiGLU MLP block the down projection is the residual bottleneck — its output is added straight into the residual stream that every deeper layer reads. Error here directly corrupts what everything downstream sees.

• gate_proj and up_proj: base precision. These are the expansion and gating paths; their outputs are multiplied element-wise inside the activation. A modest error there is partly washed out by the gating, so they tolerate the base width.

• Never quantized, carried in BF16: the 34 linear-attention layers, the learned sparse indexer, the hyper-connection arrays, the norms, embed_tokens, lm_head, and the entire vision tower. These were not FP8 in the upstream release, and they stay at full precision.

Bits round up to the nearest MLX-supported width, {2,3,4,5,6,8}. Concretely, on GLM-5.3-Flash — 320B total / 18B active, 288 routed plus 1 shared expert with top-8 routing, 45 layers — the 4-bit build gives the shared expert 6 bits, every down projection 5 bits, and the gate and up projections 4 bits. The 6-bit build rounds the down projections up to 8 bits. Group size is 64 for the 4- and 6-bit builds, 32 for 2- and 3-bit, and the shared expert always uses 64.

Is the shared-expert rule worth it? At 2-bit base the shared expert sits at 4 bits and at 6-bit base it sits at 8 — in both cases the extra two bits cost memory that the routed experts could otherwise have used, and the model card's own numbers, discussed below, suggest the trade pays off. It is the same reasoning that makes the 2bit-lite build worth shipping at all: the always-active expert is the one place a little extra precision buys the most.

The selection rule: _scale_inv as a free sensitivity signal

The bit-allocation policy assumes you already know which tensors are candidates. Picking that set is where OrcaSAQ is cleverest, because the rule is mechanical and needs no data: a tensor is re-quantized if and only if the FP8 release shipped it with a _scale_inv companion.

Why that works: the upstream GLM-5.3-Flash base is FP8 — block-wise e4m3, 128×128 blocks, with a dynamic activation scheme. Block-wise FP8 quantization stores a per-block scale and its inverse alongside the weight; the presence of _scale_inv in the checkpoint is a durable marker that the tensor went through the quantization path upstream. The upstream release has already told you which tensors are safe to quantize — no Hessian, no calibration corpus, no forward passes.

For GLM-5.3-Flash that set is the MoE and dense-MLP linears, plus the four projections of every deepseek_sparse_attention block — q_a_proj, q_b_proj, kv_a_proj_with_mqa, and o_proj — across the 11 sparse layers at depth 3, 7, 11 … 43, plus the MTP block, 12 × 4 = 48 tensors. Everything else never carried the marker and stays BF16: the 34 linear_attention layers, the sparse indexer, and the vision tower. The MTP layer — layer 45 — is included inside the quantized weights rather than exported as a separate module.

The point worth stealing is the trick itself. A model release that quantizes its weights upstream has already done a huge amount of the work of deciding what can be quantized; the _scale_inv marker is that decision, serialized into the file format. OrcaSAQ reads it back out. That is what makes the pipeline deterministic and transferable — any model that ships FP8 weights with scale metadata can be handled by the same rule, with no data pipeline at all.

The gotcha: per-module config, not top-level bits

If you write your own MLX quantizer — and that is the audience for this section — the single most useful thing in the model card is a warning: the top-level bits and group_size in config.json are not enough.

In total, 37,338 tensors are quantized. The assignment is recorded in config.json → quantization as per-module {group_size, bits} overrides keyed by MLX module path — for example model.layers.3.mlp.switch_mlp.down_proj. Because MLX fuses a layer's routed experts into a single switch_mlp, 173 entries cover all 37,338 tensors.

And the loader reads those entries at load time. If you quantize the whole file at the base width, every raised-precision tensor — the shared expert at base +2, every down projection at base +1 — comes out at the wrong width, and the model loads mis-shaped. The per-module map is not an optimization you can skip; it is the load path. When you write your own quantizer, emit the overrides for every tensor the policy raises, and verify them against the top-level default before you ship.

Does the policy pay for itself?

The evidence is our own, measured on one model: GLM-5.3-Flash, each build dequantized and run through the identical glm5_next forward so the only variable is the quantization. The numbers below are from the model card, and they are not vendor benchmarks or a third party's figures — take them as a single data point, not a law.

• Perplexity, versus the FP8 reference at 2.7797: 6-bit 2.7864 (+0.24%), 4-bit 2.8620 (+2.96%), 3-bit 3.0566 (+9.96%), 2-bit 4.3622 (+56.9%).

• Top-1 token agreement with the reference, same order: 97.76%, 96.13%, 92.06%, 86.56%.

The reading is exactly what the policy predicts. Everything down to 3-bit degrades gently — that is the signature of a bit budget spent on the right tensors — and 2-bit is a cliff, because below a point the role-based raises stop covering the damage. At 4-bit, +2.96% perplexity for a build that is roughly 38% smaller than the FP8 reference is a genuinely good trade, and it is the same policy, applied more aggressively, that makes the 102 GB 2bit-lite build load at all. Independent practitioners quantizing the same base report the same ordering — the top rungs near the noise floor, 4-bit real but modest — with different absolute numbers from a different evaluation corpus.

What transfers to your own MoE

The reusable reasoning, for a model that is not ours:

• Find the always-active experts. Whatever fires on every token — usually a shared or always-routed expert — gets your most generous bits. Its error is replayed everywhere.

• Find the residual bottlenecks. The projection that writes into the residual stream (typically the down projection of each MLP block) gets base +1. Error there is seen by every deeper layer.

• Leave the expansion and gating paths at base. If an output is multiplied element-wise inside an activation, quantization error in it is partly absorbed.

• Never-quantized upstream means never-quantized by you. If the base release carried those tensors in full precision, carry them in full precision.

• Use the upstream release's scale metadata as your selection rule. If the base model quantizes its weights, the scale/inverse-scale markers it leaves behind are a free map of what is quantizable — no sensitivity search required.

• Record per-module overrides. A global bit width will mis-shape every raised tensor on load. Write the module-path map.

And if you can hold a calibration set at all, use it to audit the policy — not to replace it. Run a calibration-driven quantization at the same average bits and check whether the ordering of the role prior matches what the data says. On a dense model or a brand-new architecture, that audit is the difference between a defensible default and a guess.

Where OrcaSAQ is the wrong choice

This is the section that should keep the method honest, because the calibration-free trade is real.

• When quality ceiling beats pipeline speed, and you have a calibration set. GPTQ- or AWQ-style methods adapt to the actual activation statistics of your model and your data, and at equal average bits they will usually beat a fixed role-based policy. If you are quantizing one model once and never re-quantizing, the extra hours of calibration are a one-time cost that buys measurable quality.

• Non-MoE dense models. The role prior — shared expert, gate/up/down — does not exist, so the policy loses the structure that makes it trustworthy. You are left with "everything upstream quantized stays quantized," which is a weaker claim.

• Models with no FP8 upstream release. The _scale_inv selection rule has nothing to key off. You must decide the quantizable set some other way, and the mechanical-transferability argument collapses.

• Brand-new architectures. The priors are exactly the assumptions that may not hold. A calibration-driven method would detect a fragile tensor that a role-based policy missed; OrcaSAQ will not, because it never looks.

• Sub-3-bit targets. The policy does not save you. At 2-bit the model is at +56.9% perplexity regardless of where the extra bits went; the 2bit-lite build exists for fit, not quality.

• When you need guarantees. Per-tensor guarantees, quantization-aware training (QAT) budgets, or the best possible quality for a fixed size with no regard to pipeline cost are all calibration territory.

The bottom line

Sensitivity-aware quantization is the practice; OrcaSAQ is one deterministic, calibration-free recipe for it. The durable lessons are the shared-expert and down-projection raises, the mechanical _scale_inv selection rule, and the per-module config that the loader actually reads. For a 320-billion-parameter MoE like GLM-5.3-Flash, that recipe produces a 4-bit MLX build at +2.96% perplexity — and the orcarouter/GLM-5.3-Flash-MLX repo ships the same policy at 2, 3, 4, and 6 bits, with a separate 2bit-lite build for 128 GB machines. Our GLM-5.3-Flash-MLX walkthrough covers which build to run on which machine, step by step.

If your priority is the last bit of quality at a fixed size and you can assemble a calibration corpus, use the calibration-driven tools and let them adapt. If your priority is a reproducible, fast, data-free quantization that transfers to the next architecture — or you simply do not want to build a data pipeline at all — the role-based policy is a defensible default. And if you would rather not quantize in the first place, the full-precision GLM-5.3-Flash is served through OrcaRouter as z-ai/glm-5.3-flash. The choice is about how much pipeline you are willing to run, not about whether sensitivity-aware quantization is worth doing.

Would rather not quantize anything at all? z-ai/glm-5.3-flash is the full-precision model served on OrcaRouter at provider pricing, 0% markup.