Kimi K3.1 Pro Cut Its Own Cyber Capability by 9.5 Points. The Other Two Labs Chose Differently

Moonshot did something on 24 October that labs usually do not: it shipped a model weaker than the one it could have shipped, and published the exact cost. On xdev-bench-hard at pass@3, a checkpoint trained without the company's cyber risk-reduction measures scores 90.5%. Kimi K3.1 Pro, the model you can call, scores 81.0%. The 9.5-point gap is the price of removing cyber-related data from mid-training and from reinforcement learning, and it is the clearest statement any of the three frontier labs has made this year about what it is willing to trade for capability.

Every major lab now has a cyber policy. They differ in mechanism rather than intent, and the mechanism determines what you actually get when you call the API. Moonshot subtracts the capability from the weights. Anthropic redirects the request to a different model. OpenAI gates access behind a risk rating. For a security team — or anyone whose workload brushes against offensive tooling — the three approaches produce three different kinds of surprise.

What Moonshot removed, and what it kept

The launch material is unusually specific. Cyber-related data was removed from both mid-training and the reinforcement-learning stage, while general coding tasks were retained. The company's stated principle is not to release open weights that would extend the absolute frontier of cyber capability, and it describes the shipped model's offensive capability as "by design, below what our pipeline can produce" and behind what frontier proprietary models have offered publicly for at least three months.

Two details in that policy matter more than the headline number.

The first is that the reduction is relative to Moonshot's own pipeline, not to the market. A model capped at 81.0% still scores above Kimi K3's 33.3% by a wide margin, and above GLM-5.3's 57.1% on the same benchmark. The cap moves a frontier model down, not out of the category.

The second is that Moonshot says it will "progressively raise" the cyber capability available in open weights. That is a schedule, not a fixed line: evaluations that assume today's ceiling may not hold at the next release, and a self-hosted copy inherits whatever ceiling existed at its release date.

What the benchmark actually asks for

Moonshot's documentation describes the evaluation in enough detail to judge it. xdev-bench-hard is built from recently disclosed vulnerabilities in software including the Linux kernel, Node.js, JavaScriptCore, Exim and VirGL, with mitigations enabled. Each task supplies the vulnerable source, a short bug hint and the compiled binary; the agent has to develop an exploit, achieve arbitrary code execution in a separate sandbox, and retrieve a secret string so the result can be verified automatically. Scoring is pass@3 over a random subset, with no hard timeouts and no output-token limits.

That is a demanding, narrow test: it measures whether a model can turn a known-vulnerable binary into a working exploit, not whether it can write a scanner or triage an alert. The 9.5-point gap between the unmitigated checkpoint and the shipped model is therefore a real measurement of a real capability difference — and a reminder that the number says little about the defensive work most teams actually do with the model.

Anthropic's alternative: keep the capability, change the answer

Anthropic's approach on Claude Opus 5.5 is serving-side routing. Its documentation says most cyber-related tasks may be rerouted to Claude Opus 4.8, and biology and frontier-model-development work may fall back to Claude Opus 5. The caller does not get an error; the response simply comes from a different model. The highest-capability tier, Claude Mythos 5.1, is available only to organisations verified through programmes such as the Cyber Verification Program.

The trade is the opposite of Moonshot's. The frontier capability exists and is served — to verified customers, under a different model id — while unverified callers silently receive an older model's answer. For a security team inside a verified organisation, this is the most capable option of the three. For anyone outside one, it is a model that quietly answers with something else on exactly the tasks where they most wanted the frontier.

OpenAI's approach: decide before you ship

GPT-6 Astra was the first OpenAI model rated "critical" under the company's Preparedness Framework, because it can independently find novel security flaws. The rollout was staged — early-access partners, then subscription tiers, then the API — and the highest-risk capabilities are restricted by rating rather than by prompt filtering. Access is decided at the account level, before any request is made.

That is cleaner than a silent reroute and slower than either alternative. It also means the answer to "can I use this model for my security work" is knowable in advance, which is worth something to a buyer who has to write a procurement document.

What the three strategies do to an evaluation

If you benchmark models on security tasks, the mechanisms distort the measurement differently, and the distortions do not cancel.

• Moonshot's cap is inside the weights. A benchmark run on the API and a benchmark run on a self-hosted copy should agree, and both measure a model that is deliberately worse at the category. The number is honest and reproducible.

• Anthropic's routing is inside the serving stack. The same request can produce different quality depending on whether it trips a safeguard, and the vendor's published benchmark figures may not reflect the routing your account receives. Our own coverage of Opus 5.5's benchmark table flagged the same issue from the other direction: the independent Artificial Analysis figure for Terminal-Bench was 59.6% against Anthropic's 66.4%, in a configuration that includes server-side safeguard routing.

• OpenAI's gating is outside the model. Once you have access, the model behaves as benchmarked; if you do not, you cannot run the comparison at all.

What a security team should take from this

Three practical conclusions, in order of confidence.

First, "cyber capability" is no longer a property you can read off a model card. It is a function of the weights you receive, the routing your account triggers and the verification programme you belong to. Any evaluation that does not state all three is measuring something else.

Second, if your work is defensive — triage, patch drafting, detection engineering — the capability cap costs less than it sounds. The 9.5 points are concentrated in adversarial, exploit-adjacent behaviour; the model's general coding and tool use, which is what defensive work mostly needs, is what the post-training run was spent improving. Moonshot reports Kimi Code Bench 3.0 long-horizon rising from 55.95% to 61.2% across the same release.

Third, the caps are moving. Moonshot has said it will raise its ceiling; Anthropic's verification programmes are expanding; OpenAI's ratings change between models. A policy that guided a deployment decision in October may be wrong by January, which is an argument for re-reading the access documentation at every model upgrade rather than treating it as a one-time gate.

Kimi K3.1 Pro is callable through OrcaRouter at $2.00 per million input tokens and $10.00 per million output, with the same end state as everywhere else: the cap is in the weights, not in what a particular provider does with your request. If your workload depends on knowing which model actually answered, that is a feature. If it depends on the frontier's offensive ceiling, no API on the market will give it to you without a verification programme.