Qwen3.8-27B-Uncensored-MLX on Apple Silicon: How to Pick Your Build, Load It in LM Studio or mlx-vlm, and Tame the 262K Context

The most useful thing to know about orcarouter/Qwen3.8-27B-Uncensored-MLX is that you do not have to choose a subfolder to run it. OrcaRouter's abliterated, Apple-Silicon build of Qw​en/Qwen3.8-27B — published to Hugging Face on August 17, 2026, so it is not new, just under-documented — keeps a copy of the 4-bit build at the repo root, specifically so tools that treat one repo as one model, most importantly LM Studio, load it with no configuration at all. That one decision is why this card has pulled roughly 83,000 downloads in the last month while comparable MLX conversions get a small fraction of that. Everything else about it is a real choice: which of the four precisions fits your Mac's unified memory, which runner you use, whether vision and tool calling survive at your bit-width, and whether the 262,144-token context window is worth what it costs on your hardware. This runbook walks those decisions in order. It is first-party documentation — the sizes, precisions and fidelity figures below are OrcaRouter's own, measured on this exact build, and every community number is labeled as such.

What exactly is in this repo

This is an abliterated build of Qw​en/Qwen3.8-27B — a 27B dense, hybrid-attention model (Gated DeltaNet linear attention plus full attention), natively vision-language, with thinking control, tool calling, a multi-token-prediction (MTP) head, and a 262,144-token context window. Abliteration removes the model's refusal behaviour by orthogonalizing the refusal direction out of the residual stream; if that is new to you, our primer on the technique is the better place to start than this page, which is about running the build, not the method. The weights are Apache-2.0, inherited from the base model.

Do not confuse this repo with qwen-3-8-27b-mlx, which is the same base model in MLX format without the abliteration. Readers routinely grab one when they meant the other: this one is the refusal-removed research build; that one is the plain Qw​en/Qwen3.8-27B on Apple Silicon. Check the repo name before you spend the download time.

One structural detail matters more than it looks: the vision tower, all norms, and the linear-attention conv1d stay in BF16, and only the language-model linear layers — including embed_tokens and lm_head — are quantized. That is why image understanding survives the quantization, and it is also why the on-disk sizes below are a little larger than a naive "27B at N bits" estimate: a chunk of the model never gets compressed.

The decision: which build fits which Mac

Four precisions ship as subfolders — 8-bit/, 6-bit/, 4-bit/, 2-bit/ — all MLX affine quantization at group size 64. The 4-bit build is additionally mirrored at the repo root. Pick by your Mac's unified memory first, quality second:

• 8-bit — ~27.5 GB on disk, wants a 64 GB Mac (a 32 GB machine can load it but leaves little room for anything else). Measured cosine fidelity against the BF16 source: 0.9997, effectively near-lossless. Choose it when quality is the whole point and memory is plentiful.

• 6-bit — ~22 GB, suits 24–32 GB Macs. Fidelity 0.9996, excellent. The best quality-per-gigabyte balance for most owners of a 48 GB machine.

• 4-bit — ~15 GB, comfortable on a 24 GB Mac. Fidelity 0.996, very good. This is our recommended default, and it is the copy at the repo root for that reason.

• 2-bit — ~8.7 GB, fits a 16 GB Mac. Fidelity 0.92 and, at 27B, severely degraded: repetition loops, garbled text, code and Chinese, partial vision. The card says it plainly — archival only, not for real work. A 16 GB Mac can technically load it; that does not make it usable.

On-disk size is not the memory number. Weights have to fit in unified memory alongside macOS, the KV cache and Metal's scratch buffers, so leave headroom. A community run of the 4-bit build on an M1 Pro 32 GB Mac measured ~16 GB of weights on disk but an inference peak of 18.5–21.7 GB; community runs of 8-bit MLX builds report peaks around 34–36 GB even when the weights are ~29.5 GB. The working guidance from that same community test is: 16 GB is not a real option for a 27B model, 24 GB can try 4-bit with short context, and 32 GB is the comfortable starting point. Our VRAM planning article walks the same math for the whole family.

Because the repo-wide listing totals roughly 94 GB across every precision, download only the subfolder you need with hf download orcarouter/Qwen3.8-27B-Uncensored-MLX --include "4-bit/*" --local-dir ./Qwen3.8-27B-Uncensored-MLX — substituting your chosen precision. The root 4-bit copy is a duplicate of the 4-bit subfolder, so you never need to fetch both.

Path one — LM Studio, zero configuration

The root 4-bit copy exists specifically so LM Studio can treat the whole repo as one model. Search the model ID, select the precision you want (the root copy is 4-bit), and the app handles the rest. Three gotchas, all from our card, and they are the entire difference between this working and failing:

• The repo is gated. Anonymous downloads get HTTP 401. Accept the terms on the model page, then paste a Hugging Face read token into LM Studio's Settings → Integrations → Hugging Face. Skip this and the load simply fails.

• Turn off KV cache quantization. MLX vision models do not support it on this architecture and loading fails during initialization if it is enabled (tracked as mlx-engine#286). This is the opposite of the advice for the GGUF builds, where quantizing the K/V cache is a legitimate VRAM saving — the two formats are not interchangeable here.

• Update the runtime. qwen3_5 architecture support landed in mlx-vlm 0.6.x; an older bundled runtime cannot load these weights at all. LM Studio's "Likely too large" badge, by contrast, is only a RAM warning, not an error — ignore it if your unified memory clears the guidance above.

Which precision in LM Studio: 8-bit is ~29.5 GB on disk and wants a 64 GB Mac; 6-bit ~22 GB suits a 48 GB machine; 4-bit at ~16 GB is the right choice on a 32 GB Mac. If you want the llama.cpp / Ollama route on other hardware, our local-running guide for the uncensored model covers that path separately.

Path two — mlx-vlm and mlx-lm from a subfolder

The command-line route gives you the precision directly and an Open​AI-compatible server for agent tools. Requirements: pip install -U mlx-vlm (mlx-vlm ≥ 0.6.13 and mlx ≥ 0.32; the Metal backend is selected automatically on Apple Silicon). After the subfolder download above:

python -m mlx_vlm generate --model ./Qwen3.8-27B-Uncensored-MLX/4-bit --prompt "Explain quantum entanglement in one sentence." --max-tokens 256

Add --image path/to/image.png for vision input, or serve it:

python -m mlx_vlm server --model ./Qwen3.8-27B-Uncensored-MLX/4-bit --port 8080

For agent frameworks that speak Open​AI-compatible APIs, mlx_lm.server is the lighter path — uv tool install mlx-lm then mlx_lm.server --model "orcarouter/Qwen3.8-27B-Uncensored-MLX", which serves at http://localhost:8080/v1 with api type openai-completions. Our card carries self-reported community configs pointing Pi, Hermes and OpenClaw at that endpoint. Those are user-supplied setups we have not reproduced, so treat them as starting points, not guarantees.

Vision survives the quantization

Because the vision tower and the conv1d stay in BF16, image understanding is preserved at 4/6/8-bit. On our probe image — shapes, colors, position, background and in-image text — the 4/6/8-bit builds described everything correctly; 2-bit was only partial. If you are choosing a build and vision matters, 4-bit is the lowest you should go. We have not published Apple-Silicon-specific vision throughput for this build, so do not trust a tok/s figure for it unless it comes from a named run on your own chip class.

The preserved vision tower is also part of the risk surface, and it is worth saying that plainly: an abliterated model that can also read images is not a text-only safety problem.

Tool calling and agent use

Tool calling is a base-model capability and it survives abliteration, which makes this build genuinely useful for red-teaming agentic systems — and genuinely dangerous if pointed at real services. The standard setup is the mlx_lm.server endpoint above, which any Open​AI-compatible agent can hit. If you do run it as an agent, understand what you have enabled: a refusal-free model with function-calling and 262K context is a different safety posture from a chat model, and the card's research-only framing exists for that reason.

The 262K context and what it really costs

The architecture gives this model an unusually cheap long context. Hybrid attention here means 48 linear-attention (Gated DeltaNet) layers interleaved with 16 full-attention layers, and only the 16 full-attention layers carry a classic KV cache — the linear layers keep a compact recurrent state instead. So 262,144 tokens costs materially less than it would on a full-attention 27B. That is a structural fact about the base model, not a promise about your Mac.

In practice the window is a capability, not a free tier. A community long-context test on an M4 Max measured roughly 63 tok/s at short context dropping to about 11 tok/s at 31K tokens — decode degrades sharply as the cache fills, and first-token latency on very long prompts is usually the bigger wall than raw throughput. Speculative and ANE-based prefill improves ingestion, not decode. Our own Apple-Silicon KV-cache cost figures for this build are not published; if you need hard numbers for your hardware, the community runs are the honest source, and the community consensus is to treat the full 262K as something you reach for deliberately, not as a default you leave on.

Speed — what is actually measured

We publish exactly one speed figure for this build and it is not Apple Silicon: ~32–37 tok/s steady-state on a single H200 via the MLX CUDA backend, which confirms the weights also run outside macOS. On Macs our card deliberately publishes no tok/s, because it depends on the chip, the precision and the runner. Practitioner numbers worth knowing, all labeled as such:

• This exact build, 4-bit, M1 Pro 32 GB: ~8.7 tok/s generation and ~41.7 tok/s prefill on a short run (community test, August 2026). An older chip, but a realistic floor.

• oMLX with native MTP on an M4 Max, stock non-abliterated checkpoint: 53.3 tok/s prose and 72.1 tok/s code, with ~273.7 tok/s prefill at 4K; raw decode without MTP ~24.6 tok/s. Practitioner-measured on a different checkpoint and runtime, so treat it as an upper bound the abliterated weights might approach rather than a promise.

• oMLX dual-ANE prefill on an M3 Ultra, abliterated oQ4e-MTP: prefill up ~17.7% at 16K and ~18.9% at 32K, and decode via Lightning MTP up to ~75 tok/s at 16K. The caveats are real: it uses private Apple runtime interfaces and approximate INT8 weights, adds roughly 4 GB of peak memory, and pushes model load time from ~3.4 s to ~28 s. That is a prompt-ingestion optimization, not a generation accelerator.

The MTP head — a free speedup if your runner supports it

This build ships the base model's multi-token-prediction drafter in the mtp/ subfolder (architecture qwen3_5_mtp), and it works with any main precision. The acceptance is lossless — with greedy decoding the output is identical to running without the drafter — so it is a genuine speedup with no quality cost when it engages. Two setup notes from our card: it requires an mlx-vlm build that includes the qwen3_5_mtp drafter (the main branch), and you must pass both --draft-model ./Qwen3.8-27B-Uncensored-MLX/mtp and --draft-kind mtp. Setting mtp_enabled alone does nothing. If your runner does not support the drafter, it is ignored and the model runs normally — nothing breaks.

Safety — read this before you run it, not after

The model card's own disclaimer is unusually direct, and this page is not going to soften it. The safety alignment of this build has been substantially removed. It will comply with harmful, unethical, offensive or illegal requests that the base Qw​en/Qwen3.8-27B would refuse, and it has no meaningful built-in guardrails. It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation and controlled experiments. If that is not what you are doing, this is the wrong model.

Two warnings from the card deserve emphasis. First, jailbreak and safety probes "succeed" trivially on an abliterated model, and that is not a passing safety evaluation — it is the expected behaviour of a model whose refusal direction has been removed. Absence of refusals is not evidence of safety. Second, the preserved vision, tool-calling and 262K context expand the attack surface into image understanding and agentic use: an uncensored model that can read screenshots and call tools is a broader risk than a chat-only refusal-removed build. Low-bit quantization adds a third layer of instability on top — at 2-bit, garbled output can even be mistaken for refusal, which is its own kind of confusing failure.

You assume full responsibility and liability for use and outputs. The Apache-2.0 licence is inherited from the base model and does not change that: it permits use; it does not make your deployment safe. Anyone deploying this to end users, minors, or any production environment must add their own moderation, safety and abuse-prevention layers first, and comply with applicable law. For researchers actually running it, the practical guardrails are: an isolated, ideally offline, environment; agent tools not pointed at real services; no exposure to end users; and review of outputs before they go anywhere.

When local is not the answer

Local is the point of this build — the weights sit on your disk, there is no per-token cost, and nothing leaves the machine. But local is also a ceiling: it is Apple-Silicon-only, you live with the quant quality, and there is no redundancy if the machine is busy. If you need a non-abliterated 27B-class model over an API for a real workload, or you want to A/B this local build against a hosted model with the same prompts, that is the gap a routing layer exists to fill. OrcaRouter puts 200+ models behind one API key at provider list price, with automatic failover and a routing DSL — a vendor price cut goes live on our side the same day it is announced, because we pass the list price through. One API, one integration, and you can compare an unproven local setup against a hosted model without standing up a second account. We do not host this MLX build — it is local weights by design — so the API is the complementary path, not a substitute for it.

The quick decision guide

• 16 GB Mac — honestly, not this model. 2-bit fits and is archival-only; you will fight memory either way. Look at a smaller uncensored model, or the community mixed 3/6-bit conversion built for 18–24 GB machines.

• 24 GB Mac — 4-bit, and keep context short; do not run vision and long context at the same time.

• 32 GB Mac — 4-bit is the sweet spot; 6-bit if you keep context tight.

• 48 GB Mac — 6-bit with headroom; 8-bit if you do not push the window.

• 64 GB and up — 8-bit, near-lossless, and 262K context within reach with the caveats above.

That is the whole runbook. The model is from August 17, 2026, it is not launch news, and it does not need to be: 83,000 downloads happened because people wanted a how-to, and this page is that how-to. Start at the repo root, load the 4-bit in LM Studio, and only leave the default build when you know what you are trading for the quality. If you came from the GGUF or FP8 side of the family, the related guides cover those paths — the decision framework here is the same, the quant math is not.

Not another build of this model — Qwen3.8-Flash-Next-Uncensored is a separate release: abliterated from Qwen3.8-Flash-Next, a 176B-stored / 6B-active mixture-of-experts preview of the Qwen4 architecture. Same abliteration technique, different weights, its own collection.

All six 27B builds — BF16, GGUF, MLX, FP8, INT8 and NVFP4 — are collected in the Qwen3.8-27B-Uncensored collection on Hugging Face.

These weights are local-only by design. For a hosted baseline to measure the abliterated build against, Qwen3.8-27B is served on OrcaRouter at provider list price with 0% markup — the stock model, safety alignment intact.