How to Run GLM-5.3-Flash on a MacBook Pro: The 2bit-Lite MLX Playbook

Running GLM-5.3-Flash on a MacBook Pro means exactly one machine: a 128 GB M4/M5 Max MacBook Pro running the 2bit-lite build of our MLX conversion, with the macOS wired-memory limit raised. If your MacBook Pro has less than 128 GB — a 36 GB M4 Pro, a 48 GB M4 Max — this playbook is not for you; jump to the last section and route the hosted z-ai/glm-5.3-flash API instead. GLM-5.3-Flash (weights at zai-org/GLM-5.3-Flash) is Z.ai's 320-billion-parameter Mixture-of-Experts model with 18B active per token — released August 26, 2026, the first natively multimodal GLM-​5 — and even the smallest build of our MLX port needs ~102 GB of weights and ~112 GB of memory. That is the entire market, and we are not going to soften it.

This is first-party documentation, not a launch write-up: the weights below are OrcaRouter's own build (orcarouter/GLM-5.3-Flash-MLX, MIT), produced with our calibration-free OrcaSAQ quantization method. We shipped it on August 26 and publicly corrected course the next day, because the original quant range did not make MacBook Pro use practical for most people — the regular 2-bit build still needed ~160 GB, which no laptop has. On August 27 we rebuilt the smallest variant as 2bit-lite specifically to fit a 128 GB MacBook Pro, and this piece is the honest account of what that buys you and what it costs. Every number marked field note below was measured on our single-H200 verification run; nothing here is a vendor benchmark. Where we repeat community practice — the wired-memory command, mlx-vlm versioning — we label it as such.

Which build fits which Mac (read this before you download anything)

The five builds in the repo each map to a minimum RAM figure. On a MacBook Pro, "which build" has exactly one answer — everything above 2bit-lite is a Mac Studio conversation:

• 6-bit — ~296 GB weights / ~320 GB RAM / near-lossless. Mac Studio with 512 GB only.

• 4-bit — ~204 GB / ~224 GB / our recommended default. Mac Studio 256 GB. This is what the repo root mirrors.

• 3-bit — ~184 GB / ~200 GB. Mac Studio 256 GB.

• 2-bit — ~145 GB / ~160 GB. Mac Studio 192 GB. This was the smallest build we shipped on day one, and it was the miss.

• 2bit-lite — ~102 GB / ~112 GB. The only build that fits a 128 GB machine — a 128 GB M4/M5 Max MacBook Pro, or a 128 GB Mac Studio or Mac mini. Any 128 GB Apple Silicon laptop (M3 Max or newer) is the same story, just slower.

The trap hidden in that ladder: the README's default download command pulls the 4-bit build (~204 GB), which is the right default for a 256 GB machine and useless on a laptop. If you follow the default command on a MacBook Pro, you end up with a model that will not allocate. The 2bit-lite path requires an explicit --include, covered below.

There is also a hard ceiling you will hit before the math above even applies. macOS does not let a process grab all of a 128 GB Mac's unified memory; the default GPU-usable ceiling is roughly 91–96 GB (community-reverse-engineered and corroborated in several write-ups of large-model MLX setups). That is below the ~102 GB of weights alone, so even 2bit-lite will not load until you raise the wired-memory limit. That step is mandatory, and it is Section 4.

Install mlx-vlm — and only mlx-vlm

GLM-5.3-Flash is a vision-language model, so it runs under mlx-vlm, not mlx-lm. This is the single most common way people get stuck, so say it early: mlx-lm is for text-only models; running a multimodal model through it produces a confusing load error. Install the VLM package at or above version 0.6.17:

pip install -U "mlx-vlm>=0.6.17"

The version pin is not decoration. glm5_next — the hybrid sparse-plus-linear-attention architecture behind GLM-5.3-Flash — only landed in mlx-vlm the same day the model shipped (August 26, 2026), and anything older will not recognize the config. The architecture matters for a second reason: this is a brand-new topology, and the first-wave ports had real correctness bugs that fluent output would not reveal. A community runtime audit of the early glm5_next MLX path found and fixed four of them — an unapplied SwiGLU clamp on every FFN block, manifold-constrained hyper-connection tensors cast to the wrong dtype (which silently corrupted the attention mixing matrix), two epsilon mismatches in the attention norms, and bf16 router logits where the reference uses float32. After the fixes, numerics matched the reference to roughly 1e-7. The takeaway for you: keep mlx-vlm current, and treat the very first release of any new-architecture port with suspicion.

Download the weights: the exclude flags, and the include you actually need

The repo root mirrors the 4-bit build, and all five variants ship as subfolders. The default command downloads the root and uses --exclude so none of the five variant folders (which together are on the order of 800 GB) come along:

hf download orcarouter/GLM-5.3-Flash-MLX --local-dir ./GLM-5.3-Flash-MLX --exclude "2bit-lite/*" "2-bit/*" "3-bit/*" "4-bit/*" "6-bit/*"

On a MacBook Pro that command is wrong for you — it gives you the 4-bit root. For a 128 GB laptop, fetch only the 2bit-lite subfolder:

hf download orcarouter/GLM-5.3-Flash-MLX --include "2bit-lite/*" --local-dir ./GLM-5.3-Flash-MLX

That is the ~102 GB download, and it is self-contained — the 2bit-lite/ folder carries its own config.json, so you point the generator straight at it. If hf is not on your PATH, install it: pip install -U huggingface_hub. Before you start, confirm you have ~110 GB of free disk and that your Mac's storage isn't the last free 100 GB — MLX memory-maps the weights, and a nearly full SSD is how these setups die mid-download.

Raise the wired-memory limit — the step everyone forgets

This is the make-or-break step on a 128 GB MacBook Pro, and the README card assumes you know it rather than spelling out the command. The default Metal working-set ceiling on a 128 GB Mac is roughly 91–96 GB, which is under the ~102 GB of 2bit-lite weights — so without this step the model fails to allocate and the load dies. The community-standard fix is a sysctl that raises the GPU wired-memory ceiling in megabytes:

sudo sysctl iogpu.wired_limit_mb=114688

That sets a ~112 GB ceiling, leaving roughly 14 GB as the OS reserve. On 128 GB machines, community values in the wild range from ~114688 (112 GB) up to ~122880 (120 GB); do not max it out — macOS needs headroom or you get window-server stutters and system hitching under memory pressure. After setting it, load the model and watch Activity Monitor: if memory pressure goes yellow, back off the number.

Two practical notes, both from community practice rather than our field notes. First, the sysctl resets on reboot and applies to the terminal session you set it in, so plan to re-run it (or script it via a LaunchAgent) — forgetting it after a reboot is the classic "it worked yesterday" failure. Second, MLX also exposes an in-process knob, mlx.core.metal.set_wired_limit(bytes), on macOS 15.0 and newer; it must stay under the sysctl ceiling, and it is the more portable option if you are scripting a notebook. Whichever you use, the effect is the same: without it, nothing here runs.

Run it: text, image, and the Python API

With the weights in place and the limit raised, generation is one command. Text:

python -m mlx_vlm.generate --model ./GLM-5.3-Flash-MLX/2bit-lite --prompt "Explain quantum entanglement in one sentence." --max-tokens 256

Image input works the same way with an --image flag — this is where the multimodal model earns its keep on a laptop, since the vision tower stays at higher precision in the OrcaSAQ layout:

python -m mlx_vlm.generate --model ./GLM-5.3-Flash-MLX/2bit-lite --image photo.jpg --prompt "Describe this image." --max-tokens 256

For a script, the Python API is the same shape mlx-vlm users know:

from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template model, processor = load("./GLM-5.3-Flash-MLX/2bit-lite") prompt = apply_chat_template(processor, model.config, "Describe this image.", num_images=1) print(generate(model, processor, prompt, ["photo.jpg"], max_tokens=256, verbose=True))

Keep --max-tokens modest. On our H200 field-note run, 2bit-lite holds roughly 10 tok/s, so a 1,024-token answer is already a two-minute wait — and long outputs are where the KV budget and the quality collapse bite (below). On a laptop you should expect it to feel slower, not faster, until a measured figure exists.

What quality you are actually getting (the measured ladder)

Here is the honest part of the playbook, and we are not going to gloss it. OrcaSAQ degrades gracefully down to 3-bit, then the cost climbs fast. Against the FP8 reference (perplexity 2.7797):

• 6-bit — perplexity 2.7864 (+0.24%), top-1 token agreement 97.76%. Near-lossless, as advertised.

• 4-bit — perplexity 2.8620 (+2.96%), top-1 96.13%. The recommended default.

• 3-bit — perplexity 3.0566 (+9.96%), top-1 92.06%. Aggressive but usable.

• 2-bit — perplexity 4.3622 (+56.9%), top-1 86.56%. A real cost.

• 2bit-lite — perplexity 6.7018 (+141%), top-1 77.19%. The smallest build, and the only one a MacBook Pro can hold.

Those are measured numbers from our own conversion pipeline. 2bit-lite is a 141% perplexity regression and a top-1 token agreement below 78% — you are running a noticeably degraded model, and the failure modes below are what that degradation looks like in practice. It is a brilliant demo and a reasonable short-form assistant; it is not a substitute for the full-precision model.

On speed, we owe you the same honesty. The published field note is H200: ~10 tok/s, stable multi-turn, everyday Q&A and short-form text fine. We do not yet have a measured MacBook Pro figure for 2bit-lite, and we are not going to invent one — Apple Silicon throughput here depends on memory bandwidth, thermals, and MLX kernel coverage for the hybrid attention, none of which we have measured for this build on this laptop. The nearest published Apple Silicon datapoint we can point you to is an independent ~450 tok/s for a 4-bit GLM-5.3-Flash build on a 512 GB Mac under a different MLX runtime — a different build, a different precision, and a desktop machine, so do not read it as your number either. Plan for slow; be pleasantly surprised if you are not.

KV cache and long context: mind the headroom

GLM-5.3-Flash advertises a 1M-token context window. That number is irrelevant on this hardware, and a how-to that pretends otherwise would be doing you a disservice. The field note is one line: give the runtime enough KV budget for your target length. On a single H200, 2bit-lite leaves about 39 GB of headroom for KV cache after the weights load. On a 128 GB MacBook Pro the arithmetic is harsher: ~102 GB of weights against a ~112 GB ceiling leaves on the order of 26 GB before macOS's own working set — and the hybrid linear-attention layers make this model's KV footprint small relative to a dense model of this size, but it still grows linearly with context.

Serving-side guidance from the wider community corroborates the point: a config that loads fine at an 8K context can run out of memory at 128K. On the laptop, keep contexts short — a few thousand tokens of Q&A or a single image — and do not try to feed it a book. The moment a workload genuinely needs long context, you are in the last section of this piece.

Long code generation is not reliable at 2bit-lite (read this twice)

This is the section a how-to that hides its failure modes would be worthless without, so it gets its own heading. The H200 field notes are unambiguous: everyday Q&A and short-form text come out fine, and long code generation is not reliable at this precision. Three failure modes reproduced on our verification run:

• Repetition loops — the model starts repeating the same lines or blocks instead of making progress, typically a few hundred tokens in.

• Missing glue code — imports, wiring, and error handling silently dropped. The generated function looks right in isolation and does not run, because the surrounding scaffolding is simply absent.

• Rewrite churn — instead of a minimal edit, the model rewrites large chunks of a file, and successive turns disagree with each other's output.

These are reproducible at 2bit-lite, and they are precisely the things a 77% top-1 agreement predicts. What practitioners who keep the laptop build around actually do:

• Use it for explanation, summarization, Q&A, and image understanding — short-form work where it is genuinely good.

• If you must ask for code, ask for one small function at a time with an explicit signature, and verify each one before moving on. Do not hand it a whole file and ask for a feature.

• For anything long — a full module, a refactor, a long agent session — route to the full-precision API. That is not a workaround; it is the correct architecture, and it is the last section.

When not to do this at all: call z-ai/glm-5.3-flash instead

The honest decision rule: run 2bit-lite on a 128 GB M4/M5 Max MacBook Pro only when you specifically want a 320B multimodal model on a laptop you can carry — offline Q&A, private documents, image understanding with nothing leaving the machine. Choose the API for everything else:

• Any MacBook Pro under 128 GB — there is no build for you. Do not try to load it; use the API.

• Long code generation or long-context reasoning — 2bit-lite fails at exactly this, reliably. Use the API.

• Quality-sensitive work — 77.19% top-1 agreement and +141% perplexity are a real drop, not a rounding error. Use the API.

• Guaranteed throughput or predictable latency — no measured laptop figure exists yet, and the field note is 10 tok/s. Use the API.

The hosted model is z-ai/glm-5.3-flash, served full precision on OrcaRouter at Z.ai's current pricing — $0.07 per million input tokens and $0.25 per million output as of this writing (launch-period pricing; Z.ai's published list is $0.15 / $0.50). That is the vendor-reported price, and it is passed through at 0% markup, so a Z.ai price cut shows up on our side the same day. You get one API key that also reaches the other 200+ models we route, and automatic failover means an experiment with this new model does not bet your production path on a single provider. In the same time it takes to download 102 GB and wait out the first cold generation, the API has already answered a week's worth of questions — and it answers them at full precision.

Local inference is worth it when the reason is local — privacy, offline work, no rate limits, a demo that runs on battery. It is not worth it on cost or quality math for any other reason, and it is not worth it at all on a machine with less than 128 GB.

FAQ

Can a 64 GB or 96 GB Mac run GLM-5.3-Flash locally? No. The smallest build, 2bit-lite, needs roughly 112 GB minimum after macOS overhead, and even a 128 GB machine must raise the wired-memory limit to load it. If your Mac is under 128 GB, the local path does not exist; route z-ai/glm-5.3-flash through the API instead.

I installed mlx-vlm and the model fails to load — what is wrong? The two usual causes, in order: a version below 0.6.17, which predates glm5_next support and will not recognize the architecture; and the wired-memory limit not being raised, because the ~102 GB build cannot allocate under the default ~91–96 GB Metal ceiling on a 128 GB Mac. Fix both (upgrade the package, run the sysctl) and it loads.

Is the local 2bit-lite build the same model as the z-ai/glm-5.3-flash API? Same underlying GLM-5.3-Flash weights, but not the same quality. 2bit-lite is a 2-bit quantization with 77.19% top-1 token agreement against the FP8 reference, and long code generation is unreliable. The API serves full precision. They are interchangeable only for short-form, quality-tolerant work.

The 30-second version

One machine, one build, one mandatory sysctl. If you have a 128 GB M4/M5 Max MacBook Pro: install mlx-vlm>=0.6.17, download only 2bit-lite/ (~102 GB), raise the wired-memory limit with sudo sysctl iogpu.wired_limit_mb=114688, and run mlx_vlm.generate — for Q&A, short text, and images. Accept that long code generation is not reliable at this precision, that there is no measured laptop throughput yet, and that a 128 GB Mac is the floor, not the standard. If any of those caveats is a dealbreaker — or your Mac is under 128 GB — the same model at full precision is one API call away on z-ai/glm-5.3-flash.

Not on a 128 GB Mac? z-ai/glm-5.3-flash serves the same model at full precision on OrcaRouter — one API key, provider pricing passed through at 0% markup, and no 102 GB download.