Qwen3.8-27B-Uncensored Benchmarks: Refusal Collapses, Capability Stays

The benchmark story of Qwen3.8 27B Uncensored (Aggressive) — the abliterated build of Qwen3.8 27B released as an FP8 checkpoint on August 15, 2026 and a GGUF build on August 16 — is not that it got stronger. It is that it stopped refusing. The model card reports harmful-prompt refusal collapsing from 64–99% on the base to 0–6% on the abliterated build with thinking off, and at or below 1.7% with thinking on, while every capability benchmark lands within ±1.3 points of the base and WikiText-2 perplexity sits at 6.96. If you are here for the qwen uncensored benchmarks, those are the headline numbers: it is not a smarter model, it is a non-refusing one.

That distinction matters, because most of what ranks for "uncensored benchmarks" is either a thin listicle of capability scores or a hype post claiming the model is "better". Neither is what abliteration does. This article walks through the actual measured data — refusal collapse, over-refusal, capability retention, and perplexity — the method that produced it, and the safety boundary you should read before you touch the weights.

What an uncensored benchmark roundup actually measures

An ordinary model review reports one axis: capability — MMLU, GSM8K, coding, reasoning. An abliterated model is interesting on a different axis, and the whole point of the "uncensored" label is that the safety axis moved. So the useful question for Qwen3.8-27B-Uncensored-FP8 is not "is it smarter?" but "what did removing the refusal mechanism cost, and how thoroughly was it removed?"

Three families of numbers have to be read together. Refusal rate on harmful-prompt benchmarks — how often the model refuses; the higher the base, the more visible the removal. Over-refusal on benign prompts — how often it wrongly refuses an innocent request; lower is better for everyone. And capability retention — whether the edit degraded the model. A benchmark summary that only prints capability scores is answering the wrong question.

The method: abliteration is a weight edit, not a fine-tune

What separates abliteration from the community "jailbreak" packs is where the change lives. A prompt-level jailbreak wraps the input; abliteration edits the weights. The method here is Arditi et al. (2024), "Refusal in Language Models Is Mediated by a Single Direction". The core finding is that refusal behavior in many models is driven by a single direction in the residual stream — estimate that direction, remove it, and the model stops refusing without being retrained.

Concretely, the card describes estimating one refusal direction from the massive-activation-masked mean difference of harmful minus harmless last-token residuals at layer 38 (round(0.6 × 64)), using AdvBench as the harmful set and Alpaca as the harmless set. The edit is an orthogonalization, W′ = W − r(rᵀW), computed in float32 and applied to 131 residual-writing matrices — the attention output projections, the linear-attention output projections, the MLP down projections, and one embedding row space. No training step runs; the vision tower is left untouched; the MTP speculative-decoding head is abliterated consistently. That is why capability survives: removing a direction is a far gentler intervention than fine-tuning the model to comply.

Refusal collapses: the numbers

Measured on the vLLM-served FP8 build and published on the Hugging Face model card (2026-08-15), refusal on harmful-prompt benchmarks falls from 64–99% on the base to 0–6% on the uncensored build with thinking off:

• AdvBench — 99.0% → 0.0%

• JailbreakBench (harmful) — 94.0% → 0.0%

• StrongREJECT — 97.3% → 2.0%

• HarmBench (standard) — 98.7% → 2.7%

• MaliciousInstruct — 99.0% → 0.0%

• SimpleSafetyTests — 64.0% → 6.0%

• ForbiddenQuestions — 73.3% → 4.7%

With thinking enabled, refusal is essentially never — 1.7% on AdvBench and 0.0% on most of the rest. The card adds one honest caveat: 30–50% of answers still prepend a short disclaimer. That is a training artifact, not a refusal — the model answers, but hedges its opening. It reads as friction, not safety.

Over-refusal falls too

The less-publicized number sits on the other side of the safety axis. On XSTest-safe — 250 benign prompts that aligned models sometimes refuse by mistake — the base model over-refuses 5.6% of the time. The uncensored build over-refuses 0.4%. Removing the refusal direction does not just stop the model from refusing harmful requests; it also stops it from refusing harmless ones it previously flagged by over-generalization. For anyone building evaluation or red-team tooling where a refusal-free baseline is the point, this is a real improvement in the tool, not a side effect to apologize for.

Capability is preserved, not improved

This is where the "uncensored = stronger" framing dies. The model card reports the abliterated FP8 build against the official base FP8 with the same scripts and settings:

• MMLU (0-shot) — 84.3% → 84.7%

• GSM8K (CoT) — 90.0% → 88.7%

• MMLU-Pro (CoT) — 77.6% → 76.8%

• CMMLU (0-shot, Chinese) — 81.4% → 80.8%

Every score lands within ±1.3 points of the base, and WikiText-2 raw perplexity comes in at 6.96 — the card's evidence that language modeling itself did not degrade. The honest reading: abliteration is close to capability-neutral on this architecture. You are not getting a better model; you are getting the same model without the refusal behavior.

The same warning applies to the wider ecosystem: "lossless uncensored" claims on community builds deserve a sceptical read. A three-way comparison on a 4B open-weight build on Hugging Face found that the technique which claimed to be lossless actually dropped TruthfulQA by about 7 points and Lambada by about 4, while its HarmBench attack-success rate hit 100%; the other two methods sat at 99.2% and 95.5%. And an aggressive trial on a different build achieved zero refusals but produced incoherent word-salad, so the shipped version backed off to gentler per-layer parameters. Abliteration results are method-specific — these numbers are specifically the FP8 build's card data.

What the card does not claim

Read the methodology before quoting the numbers. The card labels its refusal measurement as "indicative, not an LLM-judge / publication-grade number": refusal was judged by a rule-based opening-phrase classifier, with a separate "caveat" bucket for answers that complied but prepended a disclaimer. The benchmarks are text-only, run against the vLLM-served FP8 build with the language model only, and the capability suite used standard evaluation scripts. The right frame: this is the vendor's own measured data, reproducible from the published card — not an independent third-party run, and not a claim about the GGUF build, which ships quantized for llama.cpp and should be evaluated separately.

The safety boundary

This is the part that cannot be hedged. An abliterated model has had its refusal mechanism substantially removed. Concretely: it will comply with harmful, unethical, or illegal requests that the base Qwen3.8 27B would refuse, and it has no meaningful built-in guardrails. The legitimate uses are research — interpretability (studying how refusal directions are encoded), AI-safety and refusal-mechanism study, red-teaming (probing models with inputs that try to bypass their guardrails), and robustness evaluation. It is released under the Apache 2.0 license, inherited from the base model, strictly as a research artifact. You assume full responsibility and liability for how you use it and for everything it generates. Do not deploy it to end users or into production without your own safety, moderation, and abuse-prevention layers — and for any production or consumer-facing use, the standard Qwen3.8 27B is the model you actually want.

When an uncensored benchmark is the wrong thing to look up

If your question is "which model is strongest at math or coding?", you are reading the wrong numbers — the uncensored build is not a capability upgrade. If your application needs to refuse harmful requests, this model is the opposite of what you want. If you are shipping a product, do not build on an abliterated checkpoint. And if what you are actually after is a jailbreak, that is a different thing entirely — prompt-level packs sit outside the weight-level intervention discussed here and are not the subject of this article. The benchmark data in this piece exists for one narrow, legitimate question: what removing refusal does to a Qwen3.8 27B, measured.

How to access it

Both builds are on Hugging Face under Apache 2.0: orcarouter/Qwen3.8-27B-Uncensored-FP8 (block-FP8 E4M3, about 30.9 GB of weights, serving on the standard vLLM FP8 kernel path with 262K context, tools, thinking, and MTP intact) and orcarouter/Qwen3.8-27B-Uncensored-GGUF (F16 plus 12 quant tiers for llama.cpp, with Q4_K_M at 16.8 GB the recommended default for a 24 GB GPU). The model card on OrcaRouter has the pricing and the benchmark details — the uncensored build is listed there as obsidian/Qwen3.8-27B at $0.40 per million input and $4.21 per million output tokens, with 262K context, and access is gated to security researchers, red teams, and AI-safety researchers.

The bottom line

The qwen uncensored benchmarks answer a narrow question, and the answer is clean: removing refusal from Qwen3.8 27B via abliteration leaves capability within ±1.3 points while refusal on harmful prompts collapses from 64–99% to 0–6%, over-refusal drops from 5.6% to 0.4%, and perplexity holds at 6.96. Read those numbers as "does not refuse", never as "stronger". For interpretability, safety, and red-team research, that is exactly the tool the model card describes. For anything that faces a user, it is the wrong model by construction.

For legitimate research use, the weights are on Hugging Face under Apache 2.0 — orcarouter/Qwen3.8-27B-Uncensored-FP8 (the GGUF build for llama.cpp is at orcarouter/Qwen3.8-27B-Uncensored-GGUF).