Qwen3.8-27B with Unsloth: Run, Quantize, or Fine-Tune It Locally

Yes — Unsloth runs Qwen3.8 27B, and it is the fastest way to get Ali​baba's new Apache-2.0 model onto your own GPU. The whole answer in one line: open Unsloth Studio, search "Qwen3.8 27B", pick UD-Q4_K_XL (17.9GB) on a 24GB card, and chat in under a minute. Underneath that click, Unsloth shipped three things the same week the weights dropped (August 13–14, 2026): the unsloth/Qwen3.8-27B-GGUF pack built on Unsloth Dynamic V3.0 (preview) quantization, full support in Unsloth Studio and Desktop, and the v0.1.800-beta release with GGUF export, imatrix detection, and VRAM-fit improvements. It also fine-tunes the model — Unsloth's claim is 2× faster training with 70% less VRAM. Qwen3.8 27B is a dense 27-billion-parameter vision-language model whose hybrid attention keeps only 16 of 64 layers in full attention, which is why a 4-bit file fits on cards most 27B models cannot.

On sourcing: the model facts are official, from the Qwen3.8 27B model card on Hugging Face, verified August 15, 2026. The Unsloth support, quant sizes, VRAM requirements, and install commands are from Unsloth's release notes and documentation, same day. The API price is live from OrcaRouter's catalog, same day. Unsloth's "2× faster, 70% less VRAM" and "by far the strongest model for its size" are vendor claims, not independently reproduced.

What "Qwen3.8 + Unsloth" means: four different things

Unsloth is four separate things, and the keyword search results mix them up. Knowing which one you need is half the setup:

• unsloth/Qwen3.8-27B-GGUF — the quantized weight files on Hugging Face, 21 quants plus a vision projector. Built with Dynamic V3.0 (preview), not the older Dynamic 2.0 (which was announced April 24, 2025 and predates this model). This is the "download a file" route, usable from any llama.cpp build.

• Unsloth Studio — the browser app you install or run from a Hugging Face Space. Search the model hub, click a quant, chat with tools. No manual template or inference-parameter configuration.

• Unsloth Desktop — the local GUI app for macOS, Windows, and Linux that wraps Studio plus training. This is where the "2× faster with 70% less VRAM" fine-tuning lives.

• The unsloth Python library — from unsloth import FastLanguageModel, the QLoRA fine-tuning API used in notebooks and scripts.

Unsloth's release notes for v0.1.800-beta, titled "Release Qwen3.8 27B", say Qwen3.8 27B and the 2.4T-parameter Qwen3.8-2.4T-A95B "can now be run locally in Unsloth," that the 27B "runs on 17GB RAM via Unsloth Dynamic GGUFs," and that "you can also fine-tune Qwen3.8 27B in Unsloth." The same release adds NVFP4 quants, checks disk space before GGUF exports, detects real GGUF imatrix support in Studio, and makes inference up to 10% faster on GGUF with a tunable VRAM limit.

Pick your quant by VRAM, not by vibes

Unsloth's memory table is in total RAM plus VRAM (or unified memory), and it is the official guidance. A 4-bit Qwen3.8 27B needs 17–19GB; a 24GB RTX 4090, RTX 5080, or a 24GB unified-memory Mac is the realistic floor for a comfortable 4-bit setup. The three picks that cover almost everyone:

• 12GB → UD-IQ2_XXS at 9.0GB — the only file that fits whole; expect visible quality loss. Make this choice knowingly.

• 16GB → UD-Q3_K_XL at 13.4GB — the best 3-bit file, per Unsloth's own picker.

• 24GB → UD-Q4_K_XL at 17.9GB — the recommended 4-bit file and the default answer of this article.

• 48–64GB → UD-Q8_K_XL at 31.5GB — near-lossless on a workstation or dual-GPU setup.

The full ladder, from the unsloth/Qwen3.8-27B-GGUF file list and Unsloth's docs, both verified August 15, 2026: UD-Q8_K_XL 31.5GB, UD-Q6_K_XL 25.9GB, UD-Q5_K_XL 20.2GB, UD-Q4_K_XL 17.9GB, UD-Q3_K_XL 13.4GB, UD-Q2_K_XL 10.7GB, UD-IQ3_XXS 11.9GB, UD-IQ2_M 10.3GB, UD-IQ2_XXS 9.0GB. Standard K-quants (Q4_K_M 17.1GB, Q8_0 29.0GB) are there too, and the pack ships a vision projector (mmproj-BF16, 931MB) so images and video work if you load it. Unsloth calls the whole ladder "Dynamic V3.0 (preview)" — newer than the Dynamic 2.0 you will see in older write-ups, which was built for models released over a year earlier.

Run it with Unsloth in three minutes

The one-click route: on macOS or Linux, curl -fsSL https://unsloth.ai/install.sh | sh, then unsloth studio -p 8888 and open http://127.0.0.1:8888. On Windows, irm https://unsloth.ai/install.ps1 | iex. If you want zero installation at all, Unsloth's Studio is also published as a Hugging Face Space — open it in a browser, search "Qwen3.8 27B", and pick a quant by the memory you have. Studio auto-tunes the sampling parameters and chat template, offloads to RAM when VRAM runs short, and detects multi-GPU setups — the "no-config" part is real, and it is the fastest way to be running this week's model.

The files-first route, if you already use llama.cpp: download one quant and run it directly. These are Unsloth's own commands, verified against their docs:

hf download unsloth/Qwen3.8-27B-GGUF --local-dir unsloth/Qwen3.8-27B-GGUF --include "*UD-Q4_K_XL*"

./llama.cpp/llama-cli --model unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_XL.gguf --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0

Those sampling values are the model card's recommended thinking-mode settings. Swap the --include glob for *UD-Q3_K_XL* on a 16GB card, and add --mmproj pointing at the pack's mmproj-BF16.gguf if you need vision. One gotcha: llama.cpp must be a current build — the file registers the new qwen35 architecture, and anything from before release week refuses to load it.

Fine-tune it with Unsloth

Fine-tuning is where Unsloth earns its reputation: the library claims 2× faster training with 70% less VRAM versus stock Transformers + PEFT, which is what makes QLoRA on a 27B feasible on a single consumer card. The fine-tuning entry point is the plain unsloth/Qwen3.8-27B repository (safetensors, 28B files page) rather than the GGUF pack. The standard Unsloth QLoRA pattern, applied to this model:

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained("unsloth/Qwen3.8-27B", max_seq_length=8192, load_in_4bit=True)

model = FastLanguageModel.get_peft_model(model, r=16)

then a standard trl.SFTTrainer loop on your dataset. That snippet is the stock QLoRA pattern from Unsloth's docs — the training-benchmark claims are Unsloth's own and not something I reproduced — and two caveats apply: Unsloth's kernels patch the text transformer layers, so plan to handle the vision encoder separately, and keep the unsloth package at the current release because this architecture is days old and version mismatches fail loudly.

What Unsloth Studio handles for you

Running a GGUF by hand means threading the needle on context size, RAM offload, and tool calling. Studio collapses that into defaults: it sizes context from the memory that is actually free, unloads idle vision models to free VRAM for chat or training, detects multi-GPU layouts, and wires up web search, code execution, and agent connections (Claude Code, Codex, MCP) out of the box. The v0.1.800-beta release also tightened the parts that hurt in practice: GGUF exports now check disk space before starting a long merge instead of failing halfway, Studio detects whether the GGUF it is exporting actually supports imatrix (so you do not waste a run), and the memory guards no longer over-reserve GPU memory. For a model that is three days old, "no-config" is doing real work.

Local free vs API paid: the honest economics

The core difference between Unsloth and an API is not quality — it is who owns the hardware. Unsloth is the free route: zero marginal cost per token, nothing leaves your machine, no rate limits, and full control of the weights. It is not free in absolute terms: you supply the GPU and electricity, and a 2-bit file on a 12GB card is a compromised experience. Qwen3.8 27B is dense and vision-capable, so 4-bit is 17.9GB of weights before context; the machine is the price of admission.

The API route exists because the weights are Apache-2.0, so anyone can host them at near-zero marginal cost. Qwen3.8 27B is in OrcaRouter's catalog today at $0.33 per million input tokens and $2.40 per million output, the model self-hosted on our own infrastructure — no vendor price to pass through — with a 262K-token context window and text, image, and video input. Because the weights are open there is also a rate-limited free tier that charges $0 per request. A representative 10-million-token month at a 70% input share runs about $9.51 on the paid tier. If you already own a 24GB card, local wins on cost; if you do not, renting tokens at that rate beats buying a card for a side project, and the free tier is the honest way to test the model before you commit to any hardware.

When Unsloth is the wrong answer

Unsloth Studio and the GGUF pack are the right call for a single machine. They are the wrong call when:

• You own a Blackwell RTX 50-series card. The unsloth/Qwen3.8-27B-NVFP4 quants are roughly 1.5× faster than BF16 in the same VRAM and use an FP8 KV cache for longer context. That path runs through vLLM or SGLang, not a GGUF and not Studio.

• You need the full 262K native window or the 1M YaRN extension in production. A dense vision model at long context is heavy; Unsloth's context sizing will happily run shorter for you, but a serving stack with proper KV management beats a local app.

• You are serving many users. Unsloth is a local app and a fine-tuning library, not a multi-tenant server. For concurrency, autoscaling, and uptime guarantees you want a hosted endpoint.

• You have no GPU at all. An honest answer: a 2-bit 27B on a 12GB card is noticeably worse than the same model served properly. Use the free API tier first; if that is good enough, buy hardware when the use case is proven.

• You want to fine-tune the vision side. Unsloth's speedups target the text transformer layers; a full multimodal fine-tune needs a different stack.

Bottom line

Qwen3.8 27B with Unsloth is a solved problem as of this week: install Studio, pick UD-Q4_K_XL for a 24GB card (UD-Q3_K_XL for 16GB, UD-IQ2_XXS for 12GB), and the model runs with no config and no per-token bill. The fine-tuning path is real and Unsloth's speed claims are the reason 27B QLoRA is practical on one card. Know that the quant ladder is Dynamic V3.0 (preview), not the Dynamic 2.0 older articles describe; keep llama.cpp current; and if you do not own the hardware, the same weights are a $0.33/$2.40 API with a free rate-limited tier — so there is no reason to run a 2-bit file you are not happy with. Local is the free route, and for the first time in this weight class it is also the easy route.