Title card for MAI Code 2 vs SWE-2. MAI Code 2 is shown as Microsoft's model inside GitHub Copilot with no API, priced at $0.50 input and $2.50 output per million tokens, with no published pass rate and no weights. SWE-2 is shown as Cognition's model inside Devin with no per-token price, scoring 73.0 percent on DeepSWE 1.1, 92.8 percent on Terminal-Bench 2.1 and 27.3 percent on Terminal-Bench 4.0.
Guides & Insights

MAI Code 2 vs SWE-2: The Same Two Benchmarks, Published by Only One of Them

Author

jinhao.song

Date Published

Latest models · 20View all models →
Benchmarks: Artificial Analysis · updated daily
Back to all posts

MAI Code 2 and SWE-2 name almost exactly the same benchmarks in their launch material, and only one of them tells you what it scored. Microsoft's 17 November 2026 post lists DeepSWE 1.1, Terminal Bench 2.1, SWE-Atlas and FrontierCode v1.1 and then reports ratios: 96% of Claude Opus 5.5's performance at 5% of the cost. Cognition's SWE-2 post, published on 10 September 2026, lists the same first two benchmarks and prints a number for each: 73.0% on DeepSWE 1.1, 92.8% on Terminal-Bench 2.1, 27.3% on Terminal-Bench 4.0. That third number is the one to keep, because it is the edition Microsoft did not report — and it shows what happens to a coding model when the terminal benchmark gets harder.

Both models also share a distribution constraint that makes them unusual in this bracket: neither has a public per-token API. MAI Code 2 is reachable by selecting it in a GitHub Copilot client. SWE-2 runs inside Devin. Two models you cannot call from your own code, one of which publishes its numbers.

The benchmark edition nobody leads with

Cognition's table is the most useful artifact either vendor has published, and not because SWE-2 wins anything on it. It is useful because it shows the same model measured on two editions of the same benchmark.

• Terminal-Bench 2.1 — SWE-2 at 92.8%, against 91.4% for Claude Fable 5.1, 89.9% for GPT-6 Astra and 88.3% for Kimi K3

• Terminal-Bench 4.0 — SWE-2 at 27.3%, against 55.8% for Claude Fable 5.1, 57.9% for GPT-6 Astra and 21.5% for Kimi K3

• DeepSWE 1.1 — SWE-2 at 73.0%, against 74.1% for GPT-6 Astra, 72.7% for GPT-5.6 Sol and 67.4% for Claude Fable 5.1

• FrontierCode 1.1 Main — SWE-2 at 50.0%, against 53.3% for GPT-6 Astra and 50.9% for Claude Fable 5.1

Read the first two lines together. SWE-2 is second on 2.1 and last-but-one on 4.0, a fall of roughly 65 points, and it is not alone: Kimi K3 drops 67 points across the same pair. The models that hold up are the ones tuned for long-horizon work. This is not a quirk of one lab's harness — it is the benchmark edition getting harder, and it is precisely why "we ran Terminal Bench 2.1" is a weaker claim in November 2026 than it was in September.

Microsoft ran Terminal Bench 2.1. Microsoft published no score on it. Microsoft published nothing on 4.0. Those three facts together mean a buyer cannot tell which kind of coding model MAI Code 2 is: the kind that holds up when the terminal tasks get longer, or the kind that does not.

What each vendor actually disclosed

The disclosure gap is wider than the models' prices, so it is worth stating plainly.

Cognition published a seven-model table with four benchmarks, a stated comparison set, a note that rival models were run in Cognition's own CLI at their best reasoning effort, and an honest disclosure of something it excluded: Claude Fable 5.1 at maximum effort scored 50.3% on FrontierCode at $12.83 per task, below its own medium-effort result, so it was dropped from the chart. A vendor telling you which datapoint it removed is a vendor giving you the tools to discount its table.

Microsoft published the benchmark names, the harness — MSBench, its internal evaluation platform, through the GitHub Copilot CLI with subagents disabled and every model at maximum reasoning effort — and the ratios those benchmarks average to. It published no pass rate for MAI Code 2 on any of the four, no parameter count, no output ceiling, and no comparison table against named competitors beyond the two ratios. Artificial Analysis has no row for the model as of 18 November 2026, so there is no third-party reading to substitute.

One of these disclosures lets you reproduce the reasoning. The other asks you to accept a conclusion.

Neither model is an endpoint, and that changes the purchase

SWE-2 has no published per-token price and no public API. Cognition's post says it is available in Devin Desktop and CLI, with rollout to Devin Web and Fusion, and the only cost figures in it are benchmark costs per task — the kind of number you quote to show a rival is expensive, not the kind you bill against.

MAI Code 2 does have a price card, published by GitHub: $0.50 per million input tokens, $0.025 cached, $2.50 output, promotional until 17 May 2027, then $1.00 / $0.05 / $5.00. It has no API surface of its own. MAI Code 2 Flash, the smaller sibling, is the only model in the family with one — a limited-access preview in Microsoft Foundry through an OpenAI-compatible Responses API.

• Price — MAI Code 2 at $0.50 / $0.025 / $2.50 per million tokens, promotional to 17 May 2027, against no published per-token price for SWE-2

• Access — a GitHub Copilot seat across Pro, Pro+, Max, Business and Enterprise, against a Devin subscription in Desktop and CLI

• Weights — not distributed on either side

• Context — no published context figure for SWE-2, against 1 million tokens on MAI Code 2 available only in Visual Studio Code and Copilot CLI

• Published benchmark scores — none for MAI Code 2, against four for SWE-2

• Independent index row — neither model has one

The practical upshot is that both purchases are seat purchases rather than token purchases, and only one of them comes with a meter you can model. If your finance team needs a per-unit cost, MAI Code 2 gives you one and SWE-2 does not. If your engineering team needs to know whether the model improves on your repositories, SWE-2 gives you a comparison table and MAI Code 2 gives you a ratio.

Microsoft's MAI Models page announcing MAI Code 2 and Code 2 Flash under the heading 'Everyday coding at scalable prices', tagged Nov 17, 2026 and Ignite, with the MAI Code 2 model entry below it.

What each model was built to do

Microsoft's brief for MAI Code 2 is everyday enterprise work: taking code to production, deep debugging, multi-file implementation, repo-wide reasoning, UI and dashboards. The launch post is candid that the models are "explicitly not yet tuned for harder problems, like fundamental science research and very long horizon agentic work" — a sentence that sits awkwardly next to a Terminal Bench 2.1 claim, because Terminal-Bench is a long-horizon agentic benchmark. Microsoft's own positioning says it did not tune for the thing that benchmark measures.

Cognition's brief for SWE-2 runs the other way: fewer steps, cheaper rollouts, a model co-designed with the agent that runs it. The post reports mean steps per run dropping to 53 on SWE-2 medium versus 127 for SWE-1.7, medium costing 81% less than SWE-1.7, and a median of 18 steps to the first real edit against 48. Those are engineering claims about agent efficiency, and they matter more to a Devin subscriber than any benchmark row, because they decide how much of the monthly allowance a task consumes.

So the two models are tuned against different objectives again: one for the everyday task inside an IDE, one for the step count inside an autonomous agent. Neither is a better version of the other.

Where a router earns its place when neither model is routeable

Neither MAI Code 2 nor SWE-2 is on OrcaRouter's catalogue, and there is no honest way to suggest otherwise: one is a Copilot model picker entry, the other is inside Devin. What a routing layer changes is not access to those two but the cost of building the comparison. The control in any evaluation of them has to be a model you can call and measure — Claude Opus 5.5, Kimi K3.1 Pro, Qwen3.8-Max, Gemini 4 Carbon — and those sit behind one key at provider list price with 0% markup, so a benchmark run against them bills at exactly the vendor's rate.

That matters more here than in most matchups, because the interesting question about both models is comparative: does an everyday coder beat a frontier model on cost per merge, on your code? You cannot answer it by reading two vendor tables produced on two harnesses. You can answer it by running the measurable model alongside whichever of these two your team can actually reach, and counting finished tasks rather than tokens. Automatic failover is what keeps that comparison running when a provider rate limit lands mid-run, and because pricing is passed through at list, the control's cost per task is not inflated by the layer you are using to measure it.

Which one to pick, given what you can verify

• Your team is on Devin and you want fewer steps per rollout — SWE-2, and the step-count and cost-reduction numbers in Cognition's post are the ones that decide it.

• Your team is on Copilot and you want cheap tokens for routine repair — MAI Code 2 at $0.50 and $2.50, modelled at the doubled May 2027 rate as well as the promotional one.

• You need to defend the choice with a number — SWE-2, because it publishes four and the other model publishes none.

• You need long-horizon agentic work — neither, on the evidence available. Microsoft says it did not tune for it; SWE-2's own table shows the 65-point collapse when the benchmark gets harder.

• You need a per-token cost model — MAI Code 2, since SWE-2 has no published rate to model.

Cognition's 'Introducing SWE-2: Pushing the Pareto Frontier' post by the Cognition Team, dated 2026.09.10, with a lede reporting 50.0 percent on FrontierCode 1.1 Main, within one point of Fable 5.1 while being 64 percent cheaper, and describing RL scaled to the multi-trillion-parameter regime.

The thing to watch is whether Microsoft ever publishes the number that would make this comparison real. A Terminal Bench 2.1 pass rate would do it; a Terminal Bench 4.0 rate would settle it, because that is the edition where SWE-2's own table shows what a coding model looks like when the tasks get longer. Until one of those exists, MAI Code 2 is a rate card with a promise attached and SWE-2 is a table with a step count attached, and only one of them can be checked from outside the vendor.

Scoreboard comparing MAI Code 2 and SWE-2 across six dimensions. DeepSWE 1.1 not published versus 73.0 percent, Terminal-Bench 2.1 not published versus 92.8 percent, Terminal-Bench 4.0 not published versus 27.3 percent, price $0.50 and $2.50 per million tokens versus no per-token price, weights not distributed on both, and access via GitHub Copilot versus Devin only.

Compared in this article1

Detected from this article · Benchmarks: Artificial Analysis · updated daily