Qwen3.8-27B Text+Video llega a la CPU: qué cambia el PR de torchcodec de SGLang

A draft pull request in SGLang right now is titled “[CPU] Support Qw​en3.8 text+video: adding torchcodec, ffmpeg and removing pin_memory.” Strip the plumbing and it reads as a roadmap: Qwen3.8-27B — Alibaba’s Apache-2.0, 27-billion-parameter vision-language model, open-sourced five days ago — is being wired so its text+video mode runs on hardware with no GPU in the box. That is the first concrete signal that the multimodal 27B is getting a real CPU serving path in one of the inference runtimes teams actually deploy, and it matters for anyone who has been waiting to run video understanding locally without buying a 48GB card.

Nothing in that PR is merged yet — it is a draft, CI is red, and the version pins are still moving. But that is exactly why it is worth reading carefully. The model behind it is real and released, the serving ecosystem has already caught up on GPU, and this PR shows where the no-GPU path is heading. Here is what the change actually does, why CPU video inference for a 27B model is a bigger deal than it sounds, what is confirmed about Qwen3.8-27B, and where you can call it today.

The model in one paragraph

Qwen3.8-27B is the open-weight 27B of the Qw​en 3.8 generation: a dense ~27.8B-parameter vision-language model (64 layers, hidden size 5,120) with a dedicated vision tower, released under Apache 2.0 on Hugging Face and ModelScope on the evening of August 14, 2026 (Beijing time). It accepts text, images and video and returns text — natively, not through a bolted-on adapter — with a native context window of 262,144 tokens, extendable to roughly a million via YaRN. The license is the permissive one: commercial use, fine-tuning and redistribution, no revenue threshold and no separate commercial agreement, unlike the flagship checkpoint’s terms.

Two architectural details matter more to a deployer than the headline specs. The first is hybrid attention: most layers use Gated DeltaNet linear attention and only a quarter keep a full KV cache, so long-context memory is a fraction of what a conventional 64-layer dense model needs — the same trick that makes a 262K window realistic inside a single consumer GPU. The second is multi-token prediction (MTP), which ships the checkpoint with its own speculative-decoding head and measurably speeds up decode when the serving stack uses it.

It is also the video-capable one of the family. The flagship open checkpoint, Qwen3.8-2.4T-A95B, is effectively text-only in its downloadable form, and the hosted Qwen3.8-Max API adds vision on top of those weights. The 27B is the weight you can actually download and run that does text, image and video out of the box — which is why a CPU path for its video mode is worth watching.

What the SGLang PR actually changes

The pull request (sgl-project/sglang #35492) is narrow and legible. Its own description: “This PR is to support Qw​en3.8 text+video, by adding torchcodec, ffmpeg and removing pin_memory.” In practice that is three moves.

• torchcodec — PyTorch’s ffmpeg-backed video decoding library — is added to the stack, so video frames for Qw​en3.8 text+video can be decoded and preprocessed on the host CPU. The PR pins torchcodec 0.12.0 against torch 2.12, and notes that ffmpeg must stay below version 9.

• pin_memory is removed from the multimodal path. pin_memory is a GPU-transfer optimisation that pins host buffers for fast async copy to a device; dropping it on a CPU-only path is the tell that the target hardware has no discrete accelerator in the data path.

• The reproduction command launches the actual model — Qwen/Qwen3.8-27B — through sglang.launch_server on CPU with the text+video input path enabled.

Read the status labels before you build anything on it: the PR is a draft, both CI runs are failing, and it is still awaiting review from SGLang’s code owners as of today. It is a direction, not a release. The important context is that SGLang already serves Qwen3.8-27B on GPU — the cookbook covers H200, RTX PRO 6000, RTX 5090 and DGX Spark, with a dedicated lmsysorg/sglang:qwen38-27b image — so the CPU text+video path is the genuinely new piece.

Why CPU text+video matters more than it sounds

Alibaba positioned the 27B for edge AI from day one — MediaTek adapted it to Dimensity mobile chips and the C-X1 automotive cockpit the same day the weights dropped. The local-deployment story reinforced it: a Q4_K_M quantization is roughly 17.1GB, which fits a 24GB consumer GPU or an Apple Silicon Mac with 32GB of unified memory. The one thing that story could not do was video on a machine with no GPU at all. That is the gap this PR addresses.

A CPU text+video path makes video understanding deployable on plain CPU boxes — virtual machines, small servers, on-prem rack units, edge gateways — where the workload is asynchronous and throughput matters more than latency. Video captioning, content moderation, document and diagram parsing, offline retrieval over recordings: these are exactly the batch jobs that do not need a GPU, and today they still assume one for any VLM with video input.

The honest trade-off is that Qwen3.8-27B is a 27-billion-parameter model and no CPU path makes a 27B fast. Expect single-digit tokens per second on a serious AVX-class server with video frames in the context — viable for batch and overnight jobs, not for interactive chat over video. The point of the CPU path is that the option exists at all: a no-GPU deployment can run the same model’s video mode instead of dropping to a smaller vision model or shipping every frame to a cloud GPU.

Where Qwen3.8-27B runs today

The ecosystem caught up to the model fast. On the self-host side you have llama.cpp and LM Studio with GGUF packs down to a roughly 10.7GB 2-bit quant, Ollama for a one-liner, and vLLM and SGLang for proper serving. Most of that is text or image today; the video path on CPU is the part still being built.

If you would rather not run a 27B yourself, the hosted route already exists: OrcaRouter serves qwen/qwen3.8-27b with text, image and video inputs, a 262,144-token context window, and text output, at $0.33 per million input tokens and $2.40 per million output tokens. It is behind the same OpenAI-compatible endpoint as every other model on the platform — one API key, no second contract — and the platform’s automatic failover and routing DSL apply to the 200+ models on that key. A model five days old with an unproven CPU path is the textbook case for routing instead of hardwiring: you can try Qwen3.8-27B on real workloads behind a route without betting your whole call path on one serving stack, and flip between self-hosted and hosted versions without changing code.

The numbers — vendor-reported, and worth checking

Every headline figure below is Alibaba’s own, from the model card and launch materials. The model is five days old and independent reproduction is only starting to appear, so treat these as claims with a clear direction rather than verdicts. For a 27B, the coding and agent scores are the attention-getters:

• SWE-bench Pro — 61.7 (Alibaba-reported; its own table shows Claude Opus 4.6 Max at 53.4)

• Terminal-Bench 2.1 — 73.0 (agentic terminal coding)

• OSWorld-Verified — 84.3 (computer-use agent tasks)

• AndroidWorld — 81.9

• DeepSWE 1.1 — 42.2, roughly triple the previous open 27B’s 13.3

On the vision side — the side this article is really about — Alibaba reports VideoMME 87.0 for video understanding and MathVision 90.0 for visual reasoning without tools. Artificial Analysis’s early read puts the model at 52 on its Intelligence Index and 51 on its Agentic Index, competitive with much larger closed models at certain reasoning settings; those are still early scores against a five-day-old model.

One caveat matters disproportionately for anyone running the model locally or on CPU, and it is the reasoning dial. Qwen3.8-27B ships with thinking mode on and a per-request reasoning_effort setting (low / medium / xhigh). The default xhigh over-thinks badly: the now-famous first run of the 17GB GGUF took roughly 21 minutes and 22,000 reasoning tokens to draw a simple image. On CPU especially, set reasoning_effort deliberately — the difference between interactive and unusable is one parameter.

Qué ver

Three things will tell you whether the CPU path becomes real:

• Whether PR #35492 merges. It is a draft with red CI today. A merged, green version landing in a release is the point the CPU text+video path becomes runnable for the rest of us.

• Independent benchmarks. The 61.7 SWE-bench Pro and 87.0 VideoMME figures are vendor-reported. LMArena and Artificial Analysis runs over the next couple of weeks will show how much survives outside Alibaba’s own harness.

• Whether a hosted 1M-context version materialises. Native 262K is already a lot; a million-token multimodal mode is where the hybrid-attention cache savings pay for themselves.

For now the decision is simple. If you have the hardware, the 17GB Q4 GGUF plus SGLang or llama.cpp runs the model today, and the CPU text+video PR shows where the no-GPU path is heading. If you do not, Qwen3.8-27B is callable through OrcaRouter at $0.33 per million input and $2.40 per million output tokens behind a single key. Either way, the video-capable open 27B is the most interesting model to watch in the Qw​en 3.8 generation — and it is five days old.