Kimi K3.1 Pro Benchmarks: Every Score, Who Produced It, and Where the Table Is Still Blank

The most useful number in the Kimi K3.1 Pro launch is not on Moonshot's own chart. Artificial Analysis, which runs its own evaluations rather than quoting vendor tables, measures Kimi K3.1 Pro at 55 on its Intelligence Index — fourth of 215 models in its class, two points above GPT-6 Astra at 53, and three points below Claude Opus 5.5 at 58. Moonshot's launch material tells a different and more flattering story: 65.7% on Terminal-Bench 4.0, a 7.5-point lead over the 58.2% it attributes to Astra. Both numbers are real, both were produced honestly, and they disagree about who is ahead. That disagreement is the thing worth understanding before you move a workload onto this model.

What follows is every published figure for Kimi K3.1 Pro, with the source and the setting attached to each row, and a note wherever the sheet is blank. It is the same treatment this blog gives Anthropic's and OpenAI's models, for the same reason: a benchmark table is a claim about a harness as much as about a model.

The independent number, first

Artificial Analysis evaluated the model on version 4.3.2 of its index, which folds ten evaluations into a single number. The four figures that matter for a buyer are these:

• Intelligence index — 55, ranked 4th of 215 models. For reference, the median in its class is 27, and GPT-6 Astra (Max) scores 53 while Claude Opus 5.5 scores 58 on the same version of the index.

• Cost per task — $2.11 per Intelligence Index task, ranked 85th of 113 on cost. GPT-6 Astra's equivalent is $3.26. This is the number that turns a per-token price into a workload cost, and the model's low token price is partly spent back on verbosity.

• Verbosity — 150 million output tokens generated during the index run, against a median of 87 million. The evaluator calls the model "very verbose", and that matters directly: output is the expensive line on this model's own rate card at $10 per million tokens.

• Speed — 73.5 output tokens per second on the evaluator's provider, slower than the average of 82. On our own route the median time to first token is 7.15 seconds, which is the figure that decides how the model feels in an interactive coding session.

One label on that page is worth flagging: Artificial Analysis lists Kimi K3.1 Pro as a proprietary model, because as of 24 October the weights are promised but not published. That classification should change by 4 November if Moonshot keeps its date.

The vendor table, row by row, with the setting attached

Moonshot published a broad sheet of its own evaluations alongside the launch, and in several places it compares Kimi K3.1 Pro against Claude Opus 5.5 and GPT-6 Astra on the same rows. Every figure below is vendor-reported unless the row says otherwise.

• Terminal-Bench 4.0 — 65.7% for Kimi K3.1 Pro, against 66.4% for Claude Opus 5.5 and 58.2% for GPT-6 Astra. This is the headline. It is also the row with the most caveats: Anthropic's 66.4% was produced at xhigh effort, one level above its default; Kimi's own run is at the model's default of max. The two numbers come from different harnesses.

• Kimi Code Bench 3.0, long-horizon — 61.2% against 55.95% for Kimi K3. This is the cleanest predecessor comparison in the launch, and the most vendor-controlled: both the benchmark and the harness belong to Moonshot.

• BenchCAD with tools — 92.7% for Kimi K3.1 Pro, against 96.2% for Claude Opus 5.5 and 95.9% for GPT-6 Astra. Without tools, 72.8% against Opus's 73.0%; Astra did not report a no-tools figure. This is the row where the challenger visibly loses, and it sits in the middle of the launch's own 3D-creation pitch.

• Kimi Hallu Bench Hard — an 8.3% claim-level hallucination rate with tools, 5.4% without. On the same benchmark, GPT-6 Astra is better with tools (5.5 versus 8.3) and better without (4.2 versus 5.4); Claude Opus 5.5 is worse on both (15.0 and 6.8). Moonshot also reports that the rate fell from 23.4% on Kimi K3, and that responses containing at least one factual error are now 42.4% of the predecessor's rate.

• Financial and business evaluations, internal — Financial Modeling 57.6 against 50.4 for K3, Financial Research 70.4 against 58.8, Business Operation 37 against 8.9. The last is a fourfold improvement on a task where the old model was close to non-functional. Treat the magnitudes as directional; these are Moonshot's own suites.

• OfficeQA Pro v2 — 64.4, which Moonshot describes as state of the art. No second vendor's figure appears on the row.

• xdev-bench-hard, pass@3 — 81.0% for the shipped model, 90.5% for a checkpoint trained without the company's cyber risk-reduction measures, 33.3% for Kimi K3. That 9.5-point gap is the price Moonshot says it paid to keep offensive cyber capability below a threshold.

Where the table is still blank

Three gaps are worth naming because coverage of this launch has largely skipped them.

The first is in Moonshot's own API documentation. On launch day the Kimi K3.1 Pro quickstart page still carried literal placeholder strings — "[xx.x%]" — where the Terminal-Bench 4.0, Terminal-Bench-Science 0.1 and internal coding scores should be, while the marketing post carried the finished numbers. That is a documentation pipeline that shipped before the copy did, not a fabrication, but it is a reason to read the numbers from the blog page rather than the docs.

The second is Terminal-Bench-Science 0.1, where Anthropic publishes 58.7% for Claude Opus 5.5 and Moonshot's material says only that Kimi K3.1 Pro is "above K3", without a number. A row with no figure on either side of a comparison is not a row.

The third is SWE-bench. It is the benchmark most teams still quote when they talk about coding agents, and the launch post names DeepSWE v1.1 and SWE-Marathon v1.1 in footnotes without publishing scores. Whatever the reason, the most-requested number is missing.

How to read a 65.7

Two things move a Terminal-Bench score more than the model does: the scaffolding around it and the reasoning-effort setting. Anthropic's own Opus 5.5 table shows both effects within a single model — CursorBench 4.0 moves from 57.8% at xhigh effort to 52.5% at medium, while FrontierCode moves by 0.2 points between the same two settings. Kimi K3.1 Pro has three settings and defaults to the most expensive one, so a vendor run that does not state its effort level is likely to be a max run, whether or not it says so.

That is why the independent index and the vendor table rank these three models differently. On Moonshot's Terminal-Bench row, Kimi beats Astra by 7.5 points. On Artificial Analysis's ten-evaluation index, Kimi leads Astra by 2 and trails Opus by 3. Neither ordering is wrong; they answer different questions with different harnesses, and the only defensible summary is that all three models are within a few points of each other on general capability, with the price differing by five times between the cheapest and the most expensive.

What to do with the numbers

For a workload where output volume dominates — long code generation, document drafting, agentic loops that write far more than they read — the verbosity figure is the one to plan around. At 150 million tokens per index run against Astra's 60 million, Kimi K3.1 Pro produces roughly 2.5 times the output for a task, and even at a fifth of the output price that still narrows the cost advantage considerably. The $2.11 per-task figure already accounts for it, which is why it is the better planning number than the rate card.

For a workload where a point of accuracy decides the outcome, the three-point gap to Claude Opus 5.5 on the independent index is the relevant figure, and no vendor table should talk you out of it.

Kimi K3.1 Pro is callable through OrcaRouter at $2.00 per million input tokens and $10.00 per million output, with 1,048,576 tokens of context, up to 943,718 output tokens and three endpoint formats — Chat Completions, Responses and Messages — on the same key as the rest of your stack. The vendor's list price is what you pay, with no markup added, which means the arithmetic above is the arithmetic you get. The one thing this page cannot give you is your own benchmark: run a sample of your real traffic at low, high and max effort before you pick a default, because the setting moves the cost more than the model choice does.