Muse Spark Open's 150M Tokens per MW, Decoded: Most of That Number Is Cached Input

Meta released Muse Spark Open on October 15, 2026, and for the first eight days of its public life the number every write-up wanted was missing. Artificial Analysis carried the price, carried an Intelligence Index of 39, and listed no serving throughput at all. On October 23 that changed: the model appears on AA-AgentPerf at up to roughly 150 million tokens per second for every megawatt of accelerator power on four NVIDIA GB300 or B300 GPUs, and at roughly 70 million on AMD's MI355X. This is not a launch story — the weights are nine days old. It is a story about what that number measures, because the metric counts input tokens as well as output, and at the high-concurrency end of the curve almost everything it counts is a cache read.

What AA-AgentPerf is actually counting

AA-AgentPerf is not a chat benchmark with a stopwatch. Artificial Analysis built it for agent workloads, and the workload is the point: it replays real coding-agent trajectories against public repositories, sessions run up to about 200 turns, and per-request inputs range from roughly 5,000 to 131,000 tokens with a mean near 27,000. Outputs are short and lumpy — a median around 150 tokens, punctuated by longer reasoning bursts. A server's job in that workload is mostly to keep a very large, mostly unchanged prefix in cache and turn around small replies quickly.

The headline unit is defined on Artificial Analysis' datacenter page as total server throughput — input plus output tokens per second — per megawatt of accelerator power. Power is measured at the GPU, die and HBM, not at the wall and not at rated TDP, so a "megawatt" here is not a facility's electricity bill. Concurrency is not fixed either: the harness ramps load, then binary-searches for the highest number of concurrent agents the system sustains without breaking an interactivity target, scored as p25 output speed and p95 time to first token across a steady-state window where every agent has been active for at least 30 seconds.

Which means 150M is a ceiling, not a cruising speed. It is the throughput available at the concurrency where per-agent speed sits at the lowest value the target allows. The same hardware tuned for a snappier agent returns a much smaller number, and no launch slide quotes that one.

The numbers as published

• Throughput ceiling — up to ~150M tokens/s per MW on four GB300 or B300 GPUs, reported by Artificial Analysis on October 23.

• AMD — ~70M tokens/s per MW on MI355X, roughly half the NVIDIA result on the same workload.

• Within NVIDIA — GB300 showed a lead over B300 once output speeds rose, per the same report.

• Hosted speed — 142.9 output tokens/s and 0.76s to first token on the model's listing, against a 75.4 tokens/s median for comparable open-weight models.

• Intelligence — AA Intelligence Index 39, against a median of 19 for open-weight models of similar size.

• Price — $1.25 per million input tokens and $4.25 per million output, with an 88% cache discount, a $0.78 blended rate and $0.84 of spend per Index task.

Only the first three are new this week. The rest have been on the page since launch, and they pull in a different direction from the throughput number: cheap per cached token, expensive per generated one. Holding those two facts next to each other is where the useful reading lives.

The cross-check: 43 operating points on the same workload

An SGLang cookbook pull request merged on October 22 published five GB300 deployments measured on the same AA AgentPerf coding-agent scenario, one day before the leaderboard update. The five points were selected from 43 measured configurations, keeping both ends of the latency-throughput frontier and covering 81% of its area. They are the clearest public picture of what Muse Spark Open's serving curve looks like — and the PR is a framework project demonstrating its own stack, so read it as a tuned best case rather than a neutral one.

• Concurrency 1, two nodes, TP8·EP8 — 485.8 output tokens/s to a single user, 2,514 total tokens/s per GPU, p95 time to first token of 0.10s.

• Concurrency 16, four GPUs, TP4·EP4 — 208.7 tokens/s per user, 49,646 total tokens/s per GPU.

• Concurrency 64 — 97.1 tokens/s per user, 114,342 total tokens/s per GPU.

• Concurrency 160 with HiCache — 40.9 tokens/s per user, 185,382 total tokens/s per GPU, of which 1,237 are generated output.

• Cache — prefix hit rates of 96.8% to 97.6% across the five runs, which is what makes the total-token figures as large as they are.

That last bullet is the whole argument in one line. At the top of the curve, generated output is about 0.7% of the tokens the metric counts. The other 99.3% is input, and roughly 97% of that input was served from cache. The 150M-per-MW headline and the 40.9 tokens/s-per-user figure are the same run read from opposite ends. Neither is wrong, but they answer different questions, and the second one is the question an agent product actually has.

The arithmetic is at least consistent, which counts in the number's favour. Four GB300s draw on the order of five to six kilowatts of accelerator power; 150M tokens/s per MW implies roughly 750,000 to 900,000 tokens a second for the node, or 190,000 to 225,000 per GPU. SGLang's top point is 185,382 total tokens/s per GPU. Same order, same shape. That is our arithmetic against their measurement, but agreement of that kind is what tells you the headline was not assembled from a different run.

The AMD number is the one that changes a decision

Half the throughput per megawatt is not a rounding difference between vendors. It is the difference between two deployment plans, and it lines up with the cost curve SemiAnalysis publishes on InferenceX for the same model in FP4. At 128 tokens/s per user, B300 works out to $0.019 per million tokens against MI355X at $0.024. At 191 tokens/s per user, $0.027 against $0.044. At 254, $0.046 against $0.076. Those are interpolated chart points off measured runs, priced at $2.26 per B300 chip-hour and $1.50 per MI355X chip-hour.

AMD's chip is cheaper per hour and more expensive per token at every point on that curve, and the gap widens as you ask for more interactivity. That is the shape you expect when one platform's power efficiency is half the other's, and it is why the 70M-versus-150M split deserves more attention than the 150M does on its own.

Two caveats, both from the evaluators rather than from us. When AA-AgentPerf launched in June, Artificial Analysis noted that its MI355X configuration was built in-house rather than submitted by AMD, and that headroom may be larger on systems it tuned itself — so the AMD figure is a floor in a way the NVIDIA figures are not. And a throughput number is not an accuracy result. The MI35x accuracy gate covering this model's ROCm path has still not been run on hardware, and one of its two pass/fail thresholds was written as a placeholder with no AMD measurement behind it. As of this week the efficiency question has an answer. The correctness question does not.

What it does to the self-host-versus-rent sum

Muse Spark Open is listed at $1.25 per million input tokens and $4.25 per million output, with an 88% cache discount that lands a 7:2:1 blended workload at $0.78 per million. Those are the numbers a self-hosted deployment has to beat, and the comparison is where most of the internet will get it wrong this week.

Put the InferenceX figure of $0.019 per million tokens on B300 at 128 tokens/s per user next to $0.78 blended and self-hosting looks forty times cheaper. It is not. The $0.019 is priced across all tokens, and in an agent trace nearly all of them are cached input. The $0.78 blends input and output at a ratio that gives output far more weight than an AgentPerf trace does. Price your own workload's input and output separately before believing any multiple — and note that a published per-token hardware cost usually excludes the idle hours, the ops staff and the capacity you hold for peak.

The honest version: self-hosting Muse Spark Open can win, and the workload where it wins is shaped almost exactly like the one AgentPerf tests — long, stable, cacheable prefixes, short generated answers, high concurrency, a node that stays busy. A chat product with short prompts and long answers is close to the opposite, and at $4.25 per million output tokens the rented price is not the part of the bill that will hurt.

• Weights — the BF16 checkpoint is around 1.65TB; the NVFP4 build is roughly 537GB.

• Minimum sensible node — four Blackwell Ultra GPUs, the configuration the new throughput numbers come from. Meta also demonstrated the model running locally across two M5 Ultra Mac Studios, which is a different proposition: a workstation demonstration, not a serving plan.

• Licence — the Muse Model License. Free for most use, but companies above $20M in annual gross revenue need an agreement with Meta to offer it as a service, and products built on it must carry a "Built with Muse Spark Open" notice.

Where to run it, and what a router changes

OrcaRouter routes Muse Spark 1.1 and Muse Spark 1.2 and does not serve the open checkpoint, so nothing here is a claim about hosting it. What is worth knowing is that the hosted checkpoints list at the same $1.25 and $4.25 as the open model's own API price. The open weights are therefore not a cheaper way to call this model; they are a way to own the serving path. If you are not going to own it, the choice is between hosted checkpoints, and that choice is a configuration change rather than a migration.

If you are going to own it, the sensible way to find out whether your workload behaves like an AgentPerf trace is to run in parallel rather than to switch. Keep a hosted checkpoint as the fallback in your own stack, put a share of traffic on the new deployment, and move it back when the numbers disappoint. On the hosted leg of that comparison OrcaRouter passes the provider's list price through with 0% markup, so the benchmark against your hardware is not marked up on our side; automatic failover and the routing DSL are what make moving traffic a configuration change rather than a code change; and model fusion is there if you would rather compare answers across a panel than compare infrastructure.

What to watch

• Agents per megawatt — AA-AgentPerf's June launch results were published in that unit, around 91,500 agents per MW for a GB300 NVL72 on DeepSeek V4 Pro at the 20 tokens/s tier. A comparable agent count for Muse Spark Open is what would make it rankable against other models rather than merely describable.

• Meta's October 26 serving deep dive — vendor-reported, so label it as such when it lands, but the useful detail to look for is which concurrency the efficiency figure assumes.

• An AMD-submitted MI355X configuration — the half-throughput result is measured against a configuration Artificial Analysis built itself, and AA has said the headroom there may be larger.

• The MI35x accuracy gate landing with real numbers — efficiency without a correctness check is half a deployment decision, and the placeholder half of that gate is still a placeholder.

The useful reading of this week is narrow and specific. Muse Spark Open is, on the evidence now published, an unusually efficient model to serve on Blackwell at high concurrency with cacheable context, about twice as efficient there as on MI355X at the same task, and priced as a premium model when you rent it by the token. Every one of those claims has a number behind it. None of them says the model is fast in the way a user experiences fast — that number is 40.9 tokens per second at the concurrency where the headline lives.