How to Run Qwen3.8-27B-Uncensored Locally: GGUF Quants, llama.cpp and Ollama

To run Qwen3.8-27B-Uncensored locally, pull the gated GGUF repo orcarouter/Qwen3.8-27B-Uncensored-GGUF from Hugging Face, pick a quant by your VRAM — Q4_K_M (16.8 GB) for a 24 GB GPU, IQ4_XS (15.3 GB) for a 16 GB GPU, Q3_K_M (13.5 GB) when you want context headroom — and serve it with a current llama.cpp build using the --jinja flag, adding --mmproj if you need vision. The same file imports into Ollama in three commands. This is the GGUF version of the abliterated Qwen3.8 27B build, released August 16, 2026, with the native 262K context, the vision projector and the MTP speculative-decoding head preserved in every quant. It is a research tool, not an assistant: it has had its refusal behavior removed, it carries no built-in guardrails, and the license is Apache 2.0. Read the safety boundary at the bottom of this page before you download — it is the point of the gated repo.

Which GGUF to download, by VRAM

The repo ships one full-precision pair and twelve quantized files, so the download decision is really a VRAM decision. The numbers below are the file sizes read from the Hugging Face repository on 2026-08-16.

What is actually in the repo

orcarouter/Qwen3.8-27B-Uncensored-GGUF holds fifteen model files: the F16 weights split into two parts, twelve quantized GGUFs — Q2_K, Q3_K_S, Q3_K_M, Q3_K_L, Q4_K_S, Q4_K_M, Q5_K_S, Q5_K_M, Q6_K, Q8_0, IQ3_M and IQ4_XS — and the mmproj vision projector. It is the GGUF export of the same abliterated build as Qwen3.8-27B-Uncensored-FP8, re-packaged for llama.cpp-class local runtimes, and it inherits the base model's Apache 2.0 license and its native 262,144-token context.

Three properties carry across every quant. The architecture is qwen35 — hybrid attention with 48 Gated DeltaNet linear layers and 16 full-attention layers — which is why the repo refuses to load in older llama.cpp builds and why the KV cache stays small. The MTP (nextn) speculative-decoding head is preserved in all quants, K-quant and IQ alike. And the chat template is the Qwen Jinja template, which you must enable explicitly (see below) or multi-turn conversations break.

Why the KV cache stays small enough to matter

This is the detail that makes the local story work. Of the 64 layers, only 16 use full attention; the other 48 use Gated DeltaNet linear attention, which keeps no conventional KV cache. The practical effect, measured on the original runbook for this architecture, is a KV cache of roughly 2 GB at 32K context — about a quarter of what a conventional 27B would need. That is why a 16.8 GB Q4_K_M file still fits a 24 GB card with a long context window, and why the uncensored GGUF line is playable on hardware that could not host the 55.6 GB BF16 checkpoint at all.

Run it with llama.cpp

llama.cpp is the intended path. You need a current build: the trunk registers the qwen35 architecture, and older builds refuse the file outright. The command shape, from the model card's usage notes and this architecture's runbook, is:

llama-server -m Qwen3.8-27B-Uncensored-Q4_K_M.gguf --ctx-size 32768 --n-gpu-layers 99 --jinja

That gives you an OpenAI-compatible endpoint at localhost:8080. For image input add --mmproj mmproj-Qwen3.8-27B-Uncensored-f16.gguf. Swap Q4_K_M for the quant that matches your VRAM; the flags are identical.

Run it with Ollama

Ollama runs the same file by importing it from a Modelfile, which is the supported way to add a GGUF that is not in the library. In the directory holding the quant you downloaded:

FROM ./Qwen3.8-27B-Uncensored-Q4_K_M.gguf

Save that line as Modelfile, then ollama create qwen3.8-27b-uncensored -f Modelfile and ollama run qwen3.8-27b-uncensored. For vision, add the mmproj line to the Modelfile (FROM ... Q4_K_M.gguf plus mmproj path) and Ollama wires the projector in. The same --ctx and template caveats apply: set the context size you can actually fit, and remember that a refusal-removed model answers with no guardrail between you and the output.

What the refusal removal actually changed

The model card publishes a safety-evaluation suite for this build. With thinking off, refusal on the standard harmful-prompt benchmarks drops from 64–99% on the base model to 0–6% on the abliterated weights; with thinking on it refuses essentially never — 1.7% or less. Capability benchmarks stay within ±1.3 points of the base. That is the shape of every abliterated release in this line: refusals removed, capabilities intact, and the caveat that an honest fraction of answers still prepend a short disclaimer before responding — a training artifact, not a refusal.

The flip side is the whole point of the safety boundary below: a model that never refuses is not a better assistant, it is a different kind of object. It is a test instrument for measuring refusal mechanisms and for finding out whether your own guardrail works.

What to actually do with it in research

Three legitimate projects that are the sanctioned use of a refusal-removed model:

When this build is the wrong answer

If you want a normal assistant, or anything you would put in front of end users, this is the wrong model — it has no meaningful built-in guardrails, it will comply with requests the base model refuses, and Apache 2.0 does not transfer your liability away. Use the original aligned Qwen3.8-27B for that. If you want to deflect refusals on a chatbot, a system-prompt pack achieves more with less risk than a weight edit. And if you have no GPU at all but still want to study the model, the hosted card obsidian/Qwen3.8-27B on OrcaRouter serves the same uncensored line at $0.40 per million input tokens and $4.21 per million output with the same 262K context; it is gated to security and AI-safety researchers the same way the repo is, and the moderation decision still sits on your side. The model card has the pricing and benchmark details.

The safety boundary — read this before you download

This model has had its safety alignment substantially removed. It will comply with harmful, unethical, offensive or illegal requests that the original Qwen3.8-27B would refuse, and it has no meaningful built-in guardrails. It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation and controlled experiments. You assume full responsibility and liability for how you use it and for everything it generates, and you must not deploy it to end users or into production without adding your own safety, moderation and abuse-prevention layers. The authors accept no liability for misuse. Downloading the repository means you accept these conditions; the license is Apache 2.0.

This article deliberately contains no harmful prompts. The refusal numbers above come from the model card's own safety-evaluation suite, which is the correct way to measure a model of this class: run the standard refusal benchmarks, read the numbers, and design your research around what they show.

Sources and date

All facts above were verified on 2026-08-16. The file list and sizes are read from the Hugging Face repository orcarouter/Qwen3.8-27B-Uncensored-GGUF (created August 16, 2026; architecture qwen35; Apache 2.0; gated). The refusal and capability numbers are from the model card's safety-evaluation suite. The KV-cache measurement (~2 GB at 32K context) is from the OrcaRouter runbook for this architecture, How to Run Qwen 3.8 27B Locally. The hosted pricing and gating are from the obsidian/Qwen3.8-27B model page, checked the same day. Quantized-file sizes are decimal GB as stored on Hugging Face.

For legitimate research, the weights are on Hugging Face: Download from Hugging Face — no credit card, live in 60 seconds.