SWE-2 Flash vs North Mini Code: The Benchmark Inside the Benchmark

Here is an uncomfortable pairing. SWE-2 Flash is a 276B-parameter post-train released on 4 November 2026 with eight benchmark rows and no way to call it. North Mini Code is a 30B-parameter model Cohere shipped in June 2026 under Apache 2.0, with a benchmark table that overlaps SWE-2 Flash on precisely zero rows — and an independent score that SWE-2 Flash does not have at all. On paper the newer, larger model wins everything. On evidence, the older, smaller one is the only one of the two a third party has measured.

That inversion is the whole article, and it is more useful than a score comparison, because the score comparison cannot actually be made.

Two MoEs with opposite parameter budgets

Both are sparse mixture-of-experts models built around the same cost argument: pay for active parameters, not total ones. They sit at nearly opposite ends of the range.

• Total parameters — SWE-2 Flash 276B vs North Mini Code 30B

• Active per token — SWE-2 Flash 12B vs North Mini Code 3B

• Expert layout — SWE-2 Flash inherits Inkling-Small's structure; North Mini Code routes to 8 of 128 experts

• Context — SWE-2 Flash unpublished vs North Mini Code 256K, with 64K max output

• Maximum output — SWE-2 Flash unpublished vs North Mini Code 64K

• Licence — SWE-2 Flash withheld vs North Mini Code Apache 2.0 in BF16, FP8 and W4A16

The nine-fold difference in active parameters is the number that explains everything else. SWE-2 Flash can afford to be a bigger, slower-to-train model that runs cheaply per token because only 12B of it fires. North Mini Code runs on a fraction of that, which is why Cohere can put it on a laptop-class deployment and why its benchmark ceiling is lower.

Why you cannot put 89.2 next to 36.0

The comparison that will circulate is Terminal-Bench: SWE-2 Flash at 89.2, North Mini Code at 36.0. A 53-point gap that would make this a non-article. It is not a valid subtraction, for a reason that is easy to miss.

North Mini Code reports "Terminal-Bench v2" at 36.0 and "Terminal-Bench Hard" at 31.1. SWE-2 Flash reports "Terminal-Bench 2.1" at 89.2 and "Terminal-Bench 4.0" at 20.0. "v2" and "2.1" are adjacent labels on what may or may not be the same task set, and neither vendor states the task count, the harness or the scoring convention in the material being compared.

The internal evidence is that these versions are wildly different instruments. SWE-2 Flash's own two Terminal-Bench rows are 69 points apart — 20.0 on 4.0, 89.2 on 2.1. When one model scores 20 and 89 on two benchmarks with the same name and different numbers, the version is the measurement. Putting a 2.x score against a v2 score and calling the difference a capability gap is the single most common error in this comparison space.

Other rows look shared and are not. North Mini Code reports SWE-bench Pro at 40.2; SWE-2 Flash reports SWE-Bench Pro v2 Hard at 89.9. Different benchmark generation and a filtered subset. North reports SWE-bench Verified at 67.6; SWE-2 Flash does not report Verified at all. North reports SciCode 38.2 and LiveCodeBench v6 70.3; SWE-2 Flash reports neither.

That leaves DeepSWE 1.1, where SWE-2 Flash reports 60.1 and North Mini Code reports nothing.

The independent number only one of them has

North Mini Code has an Artificial Analysis Intelligence Index of 27.6 and a Coding Index of 33.4. Those are third-party measurements on a consistent methodology, taken on a model anyone can download and rerun.

SWE-2 Flash has no Artificial Analysis entry, because Artificial Analysis measures models through an API and SWE-2 Flash has no API. Every figure attached to it — the 60.1, the 89.2, the 20.0, the 24.0 on FrontierCode 2.0 — is Cognition's own, produced on an undisclosed harness, and FrontierCode is Cognition's own benchmark. Nothing here says those numbers are wrong. It says that a reader has no way to distinguish a real capability from a generous scaffold.

For a 30B open-weight model from June, an AA Coding Index of 33.4 is a modest score — above GLM-4.7-Flash's 25.9, below Qwen3.6 35B-A3B's 35.2. It is also a score you can verify, which is worth more than a vendor row you cannot.

Speed, and which claim is checkable

Cohere reports roughly 199 output tokens per second on its own API, about 2.8 times the throughput of Devstral Small 2. That is a served-endpoint measurement: a request goes in, tokens come out, and anyone with access can reproduce it.

Cognition reports roughly 200 tokens per second for SWE-2 Flash against SWE-2's 60, produced by replaying shortened session transcripts at simulated rates. The two numbers are within one token per second of each other and mean entirely different things. One is a throughput measurement; the other is a demonstration of what the model would look like at that speed.

This is the pattern across the whole comparison. The older model publishes less impressive claims with more method attached. The newer model publishes better-looking claims with none.

Where North Mini Code is genuinely weak

Not everything favours Cohere. North Mini Code is a coding specialist and behaves like one: it scores 14% on GDPval-AA and 37% on τ²-Bench Telecom, both non-coding agentic evaluations where a general model would do far better. If your agent has to do anything other than write and test code — handle a customer workflow, reason over a business process — North Mini Code is the wrong tool and SWE-2 Flash's broader base is plausibly the better bet.

Its context window is published and large — 256,000 tokens with a 64,000-token output cap — though the usable window still depends on the serving stack. And a 30B model with 3B active has a lower ceiling than a 276B model with 12B active; no amount of methodological transparency closes that.

Cost, and the case for a routing layer

North Mini Code is free to download under Apache 2.0, with a hosted rate on Cohere's own API and availability through several third-party platforms. SWE-2 Flash is free inside Devin until 30 November 2026 and unpriced after that, with no downloadable form on any horizon.

Neither is in the OrcaRouter catalogue. Cohere's model is self-hostable by anyone with the hardware, and Cognition's has no endpoint to route to — so the honest framing is that this decision is made outside our catalogue either way. What a routing layer does change is the cost of being wrong. OrcaRouter passes provider list price through at 0% markup and fails over automatically, so an application written against one model ID is not an application written against one vendor's uptime. For a pair where one side may be repriced on 1 December and the other may never be callable, that portability is the part worth engineering for.

How to decide

If you want a model you can run, inspect, quantise and benchmark yourself, North Mini Code is the only option in this pair, and its published context window and 3B active budget make it practical on modest hardware. Accept the lower capability ceiling and the narrow coding focus.

If you are inside Devin and the free window is the point, SWE-2 Flash is the stronger model on its own numbers by a wide margin, and the cost of trying it until 30 November is zero.

What you should not do is put 89.2 next to 36.0 and conclude anything. Those numbers come from different instruments, reported by different companies, on a benchmark family whose version number is the measurement. The only defensible statement about this pair is the one that sounds least like a verdict: one of them has been independently scored and the other has not, and that is a bigger difference than any row in either table.