Qwen3.8-Flash-Next-Uncensored: Run the Abliterated MoE on llama.cpp and Apple Silicon

Before you download anything: Qwen3.8-Flash-Next-Uncensored is not Qwen3.8-27B-Uncensored. They share a family name and an abliteration technique, but they are different models from different weight drops, and every earlier "Qw​en uncensored" post on this blog is about the 27B. The subject here is the pair OrcaRouter published to Hugging Face on August 26, 2026 — orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF and orcarouter/Qwen3.8-Flash-Next-Uncensored-MLX — abliterated builds of Qwen3.8-Flash-Next, Alibaba's 176B-stored / 6B-active routed mixture-of-experts model that previews the Qwen4 architecture. We announced the drop as "GGUF + native MLX, up to 262K context," aimed at security researchers, red teams and blue teams. This runbook is written from our own model cards: what refusal removal does inside a routed MoE, what the 262K context claim actually costs in memory, how to serve both file lines, and where the research line sits. If you arrived wanting the abliteration primer or the 27B quant math, the earlier posts in this series cover those.

First, the gate

Both repos are gated on Hugging Face (gated: auto). No file downloads until you are logged in and have accepted the repository terms on each repo — the GGUF and MLX repos each carry their own gate. This is the most practical difference from an ungated GGUF line: the naive "just hf download" one-liner fails with an auth error before it fetches a byte. The flow is:

• Log in to Hugging Face (or sign up) and install huggingface_hub, then run hf auth login once so your token is on disk.

• Open orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF in the browser, accept the terms, then repeat for orcarouter/Qwen3.8-Flash-Next-Uncensored-MLX.

• From then on, hf download with your authenticated token works like any other repo. Every quant you pull, and the MLX weights, is a research artifact under Apache-2.0 — the same licence as the base model — and the gate is part of the deal: reading the terms is step one of using the model.

What abliteration does to a routed MoE

The technique is the Arditi-style weight edit we have covered elsewhere on this blog; the interesting part is what it does to this architecture specifically. On the dense 27B it was 131 residual matrices. On Qwen3.8-Flash-Next-Uncensored the MLX model card records abliteration applied to 149 residual-writer tensors, and the components that make this model what it is — the MoE router, the 51B n-gram embedding table, the vision tower — were explicitly never touched.

That "never touched" is the whole story for a routed model. Every token activates 10 of 512 experts; no single expert owns refusal. The refusal behaviour lives in the residual stream that composes the final output, which is precisely the direction the orthogonalization removes. So the router keeps routing the same experts, the n-gram table keeps producing the same embeddings, and what changes is what the model says once the output projection runs. The GGUF model card's published checks put the shape of that change as harmful-prompt refusal dropping from 64–100% on the base to roughly 0–3.3% on this build, benign over-refusal near 0%, and capability within ±2 points of the base on MMLU-Pro / GSM8K / CMMLU-style checks. These are the card's own numbers, self-reported rather than independently reproduced.

The MTP head: present in MLX, dropped in GGUF

Qwen3.8-Flash-Next ships a ~4B multi-token-prediction speculative head, and the two builds disagree on it. The GGUF repo excludes it — the card is explicit that these files do not include the mtp speculative-draft head — because llama.cpp's qwen4exp support does not implement MTP yet, so the head would be dead weight in the file. The MLX build keeps it, so on Apple Silicon you still get speculative decoding. The practical consequence, corroborated by practitioners running the base model: the llama.cpp GGUF lane runs without speculative decoding today, while an MTP-enabled SGLang setup more than doubles decode on the same class of hardware. If llama.cpp ever lands MTP for qwen4exp, the GGUF line gets a free speed-up — but do not buy hardware expecting it this week.

The 262K context claim, and what the KV cache costs

Native context is 262,144 tokens (YaRN-extendable toward 1M), and the constraint that actually bites is memory. The good news is the architecture keeps the KV cache small: of the 48 layers, 36 use Gated DeltaNet linear attention, which compresses history into a fixed-size recurrent state, and only the 12 full-attention layers carry a conventional KV cache that grows with sequence length.

Two community-measured data points, both on the base model, transfer directly. A 4× RTX 3090 GGUF deployment reported going from 65K to 131K context at only ~0.78 GB per card of additional KV, and a single DGX Spark ran full 262K context with a Q4-class file while keeping the model resident at ~76.9 GB of its 128 GB pool by pinning the n-gram table to CPU and mmap'ing it from NVMe. The GGUF model card's own budget line is total = file size + KV cache + the ~0.9 GB mmproj. The upshot for quant choice: at 262K the KV cache is a real line item, but the weights are the dominating one, so the same VRAM-first logic from the 27B GGUF guide applies — the difference here is the file sizes themselves, which run from IQ2_XXS at roughly 52 GB to Q5_K_M at roughly 125 GB.

The GGUF line: 13 quants, split files, mmproj, and a llama.cpp build

The GGUF repo ships 13 quant levels — IQ2_XXS, IQ2_M, IQ3_XXS, IQ3_M, IQ4_XS, Q2_K, Q3_K_S, Q3_K_M, Q3_K_L, Q4_K_S, Q4_K_M, Q5_K_S, Q5_K_M — with the IQ quants built from an importance matrix on English, Chinese and code calibration text. Every quant is multi-part, split with llama-gguf-split, so download the whole set for a quant and point the loader at the ...-00001-of-000NN.gguf part. There is no Q6_K, Q8_0 or F16 tier, and the reason is structural rather than economical: the n-gram (PLE) embedding table is one tensor too large to satisfy Hugging Face's 50 GB per-file limit at 6-bit and above, and a single GGUF tensor cannot be split across files — so the line tops out at Q5_K_M.

The other file you must not skip is mmproj-...-F16.gguf, roughly 0.9 GB: this is a vision-language model, and llama.cpp needs the projector for any image input. Qwen3.8-Flash-Next is multimodal, and so is this build — the abliteration does not touch the vision tower.

llama.cpp support is not in mainline yet. The architecture id is qwen4exp, and stock builds fail with "unknown architecture 'qwen4_exp'"; you need a build from PR #27742 (branch qwen4exp/qwen3.8-flash-next), compiled with the llama-cli, llama-mtmd-cli, llama-server and llama-gguf-split targets. From there the serving shape is the usual llama.cpp server with Qw​en-specific flags, using the quant name from the part files:

llama-server -m Qwen3.8-Flash-Next-Uncensored-Q4_K_M-00001-of-00003.gguf --jinja --mmproj mmproj-Qwen3.8-Flash-Next-Uncensored-F16.gguf -c 8192 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0

Set -c to whatever context you can fit; the card's example starts at 8192. Reasoning is on by default and returned in reasoning_content, and tool calling works through the OpenAI-compatible endpoint. The sampling values above are the model card's recommendation; practitioners on the base model use temp 0.7 / top-p 0.80 / top-k 20 with a presence penalty of 1.5 in non-thinking instruct mode, and tune reasoning depth with --chat-template-kwargs {"reasoning_effort":"medium"}.

The MLX build on Apple Silicon

The MLX repo is the native Apple-Silicon path: same weights, same refusal removal, a Metal-native runtime. It ships 4-bit as the default (~163 GB), the announced 6-bit build (~192 GB) and an 8-bit tier (~221 GB), and because the fused-3D experts and the n-gram table are kept at higher precision than the nominal tag, effective precision runs well above uniform 4-bit — the card lists ~7.85 effective bits per weight for the "4-bit" tag. This is why the repo sizes look large: the n-gram table is not crushed down the way community quants crush it.

Hardware is the binding constraint. MLX runs only on Apple Silicon (Metal), and you need unified memory for the weights — the model card's line is "a Mac with enough unified memory for the 163 GB weights (e.g. M-series Ultra)". Treat the 4-bit tier as a 192 GB-class machine and the 6-bit build as 256 GB-class. The card's tags record the full feature set — qwen4_exp, MoE, MTP, function calling, vision-language — and it is driven through mlx-vlm for image input. The MTP speculative head is included here, which is the MLX build's quiet advantage over the GGUF line.

The vision path, for those using it

Because this is a vision-language model, the multimodal path is part of the runbook rather than an extra. On llama.cpp, image input requires both the mmproj projector and a build that includes llama-mtmd-cli / llama-server. On MLX, you drive it with mlx-vlm rather than the text-only mlx-lm driver. For red teams the vision path is where interesting eval work lives: multimodal guardrails, prompt injection carried in an image, OCR against screenshots of your own systems, and adversarial images aimed at the whole pipeline. Since the abliteration covers the full model and leaves the vision tower intact, an image that would have triggered a refusal in the text head simply lands on a model with no refusal behaviour to hit. The GGUF card says vision and multi-turn tool calling were verified on this build; no separate vision eval suite is published.

What this is for, and where the line sits

This build exists for one class of work: evaluating what a refusal-removed model can do, in service of understanding and defending systems. For a red team that means testing your own guardrails against a model that will not politely decline — prompt-injection resistance, exfiltration scenarios, tool-use abuse, and the gap between "the base model refuses" and "the model actually cannot do this." That gap is the entire research value of an abliterated build: it tells you what the refusal layer was hiding, which is the difference between security by policy and security by capability. For a blue team, the same weights are the adversary's plausible baseline: if a hostile actor can download and run this, your defenses need to hold against a model that answers instead of demurring. Evaluations built on it are a lower bound on what a custom, never-aligned model could do — treat them that way, not as a ceiling.

Be clear about what removing refusals does and does not change. It changes output behaviour — the model no longer declines — and it does not change capability. No new knowledge, no new skills, no new compute; the same training, the same limits on what the model can actually produce. An abliterated model cannot engineer malware it was incapable of before; it simply answers instead of hedging, and its outputs are no more truthful for being more permissive. The card's ±2-point capability band and the collapse of the refusal numbers are the same fact seen from two sides.

Where the line sits is the gated-repo agreement, and it is not boilerplate. Legitimate use: evaluating your own systems, public vulnerability research on models, building detection and defensive evals, studying refusal mechanisms. Not legitimate: deploying this as a user-facing assistant, generating working malware or exploits against systems you do not own or lack authorization to test, fraud, weapons material. The Apache-2.0 licence and applicable law are the floor; the gate is the explicit research-purpose agreement above it. If your use case is "ship a chatbot to users," this is not your model — and that is by design, not an oversight.

How to get the served baseline

These weights are local-only on purpose: a red team's probe payloads should never transit a third-party API, and self-hosting is the point. When you want the censored, served baseline for comparison — Alibaba's Qwen3.8-Flash at $0.16 per million input and $0.47 per million output — OrcaRouter routes it at the provider's list price with 0% markup and automatic failover, so an eval harness can move between the hosted base and your local uncensored build on one key and no second contract. The open Qwen3.8-Flash-Next weights are not on our catalogue yet; when a runtime we route carries them, they land in that same one-key, list-price setup.

Start here

Decide which line matches your hardware, then read the relevant gate. On a 128 GB+ unified-memory Mac, the MLX build gives you MTP and vision in one place — accept the MLX repo terms, pull the 4-bit or 6-bit weights, and drive them with mlx-vlm. On NVIDIA, AMD, or a CPU box, build llama.cpp from PR #27742, accept the GGUF repo terms, pull a quant that fits, and do not forget the mmproj file. In both cases the refusal numbers and the capability band are the repo's own, published on the card and unreproduced as of this writing — the honest way to read them is as the vendor's measurement of its own edit, not as an independent audit. The gate, the licence, and the safety boundary above are the same text from three angles: this is a research instrument, and it is released on that condition.

All five Flash-Next builds — BF16, GGUF, MLX, FP8 and NVFP4 — are collected in the Qwen3.8-Flash-Next-Uncensored collection on Hugging Face.

Not to be confused with the 27B family: Qwen3.8-27B-Uncensored is a different model abliterated from a different base, with its own collection and its own runbooks. Same technique, different weights.

These weights are local-only by design. For a hosted baseline to measure the abliterated build against, Qwen3.8-Flash is served on OrcaRouter at provider list price with 0% markup — the stock model, safety alignment intact.