Muse Glimmer Hits 230 Tokens/s on a Single RTX 5090: SGLang Day-0 Support Is the Real Launch Story

Two hundred and thirty tokens a second is a fast number for any model. For Muse Glimmer — Meta's 30B dense, open-weight release from Meta Superintelligence Labs, which shipped on 10 August 2026 under Apache 2.0 — it is the number that separates a "runs on my GPU" model from a "serves a real agent" one. SGLang announced day-0 support for Muse Glimmer on the same day the weights went up, and its published measurement on a single GeForce RTX 5090 is 236.4 tokens per second per user at batch 1, with NVFP4 quantization and Meta's DFlash speculative-decoding drafter both on.

That one figure does most of the storytelling for this release, but it is worth slowing down over the rest of it, because the interesting part is not the headline speed. The interesting part is that the model also works out of the box on an RTX PRO 6000, on an NVIDIA DGX Spark, and on Apple silicon through MLX — and that SGLang calls the work a collaboration with Meta Superintelligence Labs rather than a community afterthought. This is the first frontier-adjacent Meta model in a long time that is designed around the serving stack, not just around the weights.

What "day-0 support" actually means here

Day-0 support in SGLang v0.5.17 means the model was not bolted on after the fact: the runtime work landed in the same release window as the weights, with a purpose-built path for the hardware Meta actually targets. SGLang's SM120 backend covers the Blackwell-desktop and workstation line — the RTX 5090, the RTX PRO 6000, and the DGX Spark all run the same optimized path — while Apple silicon gets a native MLX backend rather than a slow port.

The stack SGLang and Meta tuned for is specific. The model runs as an 18GB NVFP4 checkpoint paired with a 5GB BF16 DFlash drafter, which is the speculative-decoding companion model Meta trained for Muse Glimmer. That combination fits inside a 32GB card with room to spare, and the whole NVFP4-plus-MXFP8 mixed-quantization path lands at roughly 19.5GB resident. On the SGLang side, the release notes cite three reliability mechanisms as part of the same story: DFlash speculative decoding, RadixAttention for prefix caching, and interruptible CUDA graphs.

The number, and what it is and isn't

SGLang's published RTX 5090 numbers, all with NVFP4 and the SM120 path, break down like this:

• Single-user decode (batch 1) — 63.9 tokens/s standard, 236.4 tokens/s with DFlash speculative decoding. That 3.7x lift is the interactive number, the one that decides whether a local agent feels like a conversation or like a queue.

• Aggregate output (batch 8) — 501 tokens/s standard, 1,452 tokens/s with DFlash. This is the throughput number for a local server serving several requests at once.

• For comparison, a GGUF q4_k_m on the same GPU — the path most people will use with llama.cpp-style runtimes — reaches 72.6 tokens/s standard and 140.7 tokens/s with DFlash. The NVFP4 path is meaningfully ahead.

These are SGLang's and Meta's published figures from launch day, not independent reproductions — the honest label is vendor-and-framework-reported, and community verification will land over the coming weeks. But the pattern across the two stacks that announced numbers is consistent. In llama.cpp, Meta reported 74.9 to 233.4 tokens/s on an RTX 5090 with DFlash, a 3.1x lift; the M5 Max goes 26.6 to 50.2; the M4 Max 23.7 to 37.8. Two different runtimes, two different measurement styles, same ballpark: a 30B model decoding at 200-plus tokens per second on a single consumer GPU, and DFlash roughly tripling throughput on every Nvidia setup that has published numbers.

Why "serves" matters more than "runs"

Meta's hardware blog makes a sharper claim than the desktop numbers. On NVIDIA Blackwell Ultra — the data-center generation, not the desktop card — the blog quotes over 20,000 tokens per second per GPU at BF16/NVF4, with SGLang and vLLM both named as supported open-source stacks. That is a different league, and it is the tell for what Meta is actually building: the same weights are meant to run anywhere from a laptop to a GPU server rack, and the serving frameworks were treated as first-class from day one rather than as ports to be written later.

The reliability half of the story matters just as much as the speed. A local agent that dies mid-task because the serving layer stuttered is not meaningfully better than a cloud agent that got rate-limited. The mechanisms in the day-0 stack — speculative decoding that is interruptible, prefix caching that keeps long agent contexts cheap to re-enter, and a drafter tuned to the base model — are exactly the parts that make always-on local agents survivable. This is the "speed and reliability" the launch has been pointing at, and it is a real engineering position, not a marketing phrase.

What you can actually do with it

Muse Glimmer is a 30B dense, multimodal model — text and image input through a frozen perception encoder, text output — trained on more than 100 languages, with a 131,072-token context window and a knowledge cutoff of 4 January 2026. Its brief is the unglamorous agent work: function calling, tool use, schedule and file management, LLM-as-a-judge evaluation, and multi-step tasks with failure recovery — when a tool call fails, the model is trained to diagnose and retry rather than stop. It exposes reasoning-effort levels (low through xhigh) that you set in the system prompt, which is how you trade latency against depth on the same weights.

The 230-tokens-per-second figure is what makes all of that usable on one machine. Below roughly 50 tokens/s, an interactive agent feels like a remote debugging session; at 200-plus, it feels like a local tool. That is the entire argument for the model in one number, and it is the reason the hardware list — RTX 5090, RTX PRO 6000, DGX Spark, MLX Macs — reads like a shopping guide for the local-agent era rather than a compatibility footnote.

What it costs

Muse Glimmer itself is free — Apache 2.0 weights on Hugging Face at meta-models/Muse-Glimmer-30B, with GGUF quantizations already published for llama.cpp-style runtimes and no public Meta API. The real cost is the hardware underneath it: an 18GB NVFP4 checkpoint plus the 5GB drafter means you want a 24GB or 32GB card, or a high-end M-series Mac. If you already own that, the marginal cost of an always-on local agent is electricity. If you do not, the GPU is the line item, and that is the honest way to compare "free model" against a hosted API.

When you do make that comparison, price both sides straight. At OrcaRouter, the price you see for any of the 200-plus hosted models is the provider's list price passed through at 0% markup, so a vendor price cut is live here the same day — which keeps the local-vs-hosted arithmetic honest. Muse Glimmer itself is not on OrcaRouter today, because no inference provider is serving it yet; it ships as a download. The moment a provider does start serving it, the same key that reaches the rest of the catalogue will reach it, and automatic failover is the mechanism that makes it safe to point production traffic at a brand-new, vendor-reported model before it has an independent track record.

The bigger story behind the speed

Read the day-0 SGLang work as a signal, and the release stops being a single model. Muse Glimmer is a distilled version of Meta's proprietary flagship Muse Spark 1.2, and Meta has said the teacher's weights are coming "in the coming weeks." A 30B local agent with a tuned serving stack, a teacher model about to go open, and a $1 billion community fund announced alongside it — that is not a point release. That is the thin end of an open-weight lineup aimed squarely at the local-agent market, and the framework support landing on day one is the part that says Meta intends people to actually run it, not just download it.

The caveat to keep on the table: every speed and benchmark figure attached to this launch so far is vendor- or framework-reported. The throughput numbers are consistent across two independent stacks, which is reassuring, but independent benchmarking, an Artificial Analysis entry, and real-world reproductions are what will confirm the 230-tokens-per-second story — and they usually land within weeks of a release like this.

What to watch, in rough order of speed: the Muse Spark 1.2 open-weights drop (the strategic read on "Meta is back" depends on it); the first independent throughput reproductions on non-Nvidia hardware, where the MLX path is the one to watch; and the first inference provider standing up a Glimmer endpoint — which is the moment the "free local model" and "hosted API" arguments stop being hypothetical and become a choice you can make with one key.