Ultramarine 1 Flash vs GPT-5.6 Luna: Two Models That Never Appear on the Same Scoreboard

The first thing to notice about this pairing is what is missing from the vendor's own chart. Paper Instruments published Ultramarine 1 Flash on 14 October 2026 with six benchmark tables and a cost-quality plot covering thirty-odd model configurations. GPT-5.6 Luna is not on the plot. Not at low effort, not at medium, not at maximum. Its sibling GPT-5.6 Sol is there at every effort level; a model called GPT-6 Luna is there at 58.3 for $0.11 a task; GPT-5.6 Luna does not appear at all. And on the other side, Artificial Analysis has an entry for GPT-5.6 Luna — Intelligence Index 37, $0.18 per completed task, 115.4 output tokens per second — and no entry for Ultramarine 1 Flash whatsoever. Two models released three months apart, both aimed at cheap high-volume work, and no third party has ever measured one of them on a benchmark that contains the other.

That absence is the comparison. GPT-5.6 Luna is OpenAI's fast, cost-efficient tier from the GPT-5.6 family, shipped on 9 July 2026, carrying an independent score across a composite of ten evaluations. Ultramarine 1 Flash is a reinforcement-learning post-training of GLM-5.3 Flash, priced at $0.15 input / $0.50 output per million tokens, built to produce Excel workbooks, slide decks and Word memos rather than answers, and carrying only its maker's numbers. Everything below is about what can and cannot be concluded from that arrangement.

Where the two records actually sit

Luna's numbers are Artificial Analysis measurements on OpenAI's own API. Ultramarine's are Paper Instruments' published rates and its own published evaluations, with its context figure taken from the training configuration in its technical report.

• Independent score — Intelligence Index 37 at maximum effort, fourth of 148 models in its class, for GPT-5.6 Luna vs no independent evaluation of any kind for Ultramarine 1 Flash.

• Input price, on the rate cards — $0.20 per million tokens vs $0.15. The Paper Instruments model is 25% cheaper going in.

• Output price — $1.20 vs $0.50 per million. 2.4x, the widest gap in the pairing.

• Cached input — $0.02 per million (90% off) for Luna vs $0.03 (80% off). A cent apart, in OpenAI's favour.

• Context — 1,050,000 tokens on Luna's rate card vs 131,072 in Ultramarine's training configuration. Eight times the window, and the one line where the gap is not close.

• Maximum output — 128,000 tokens vs not published.

• Measured cost per completed task — $0.18 on the Intelligence Index for Luna (max effort) vs $0.13 on PI Knowledge Work for Ultramarine 1 Flash. Two different suites, two different kinds of task, two numbers that look comparable and are not.

• First token — 121.46 seconds for Luna at max effort on OpenAI's API, against a 1.85-second median for its price tier; not published for Ultramarine 1 Flash in any form.

The two-minute first token, and why it is a setting rather than a flaw

That 121.46-second figure deserves its own section, because it is the kind of number that gets quoted as a defect when it is actually a parameter.

GPT-5.6 Luna at maximum reasoning effort runs its deliberation before it emits anything. On the Intelligence Index it produces 115.4 output tokens per second once it starts and 150 million output tokens in total to complete the suite — 41st of 60 in its class on verbosity, described by Artificial Analysis as somewhat verbose against a median of 95 million. The Index score of 37 is measured at that maximum setting.

The model supports configurable effort, which means the same weights that take two minutes to begin can be dialled to a level that responds immediately, at a lower score. OpenAI publishes no per-effort breakdown on the Artificial Analysis record, so the size of that drop is undisclosed — a default call is somewhere below 37 by an amount nobody outside OpenAI has measured. For a fan-out step where a hundred cheap calls each draft something short, that is the right configuration. For the single hard task at the end of a chain, maximum effort is. Both are the same model and neither is misconfigured.

Ultramarine 1 Flash publishes no latency figure, no throughput figure, and no time to first token. Its technical report does disclose the operating envelope it was trained and evaluated inside — 131,072 tokens of context, a 250-turn cap on the agent loop, compaction triggered at 85% of the window or when 30,000 tokens remain — and a harness built to compact its own working memory is a harness built for long jobs. That is an inference about task shape, not evidence about speed, and the honest position is that one side of this comparison has a measured latency problem and the other side has no measurement.

What a deliverable benchmark measures that an intelligence index cannot

The reason the two models never meet is not an accident of scheduling. The benchmarks measure different things on purpose.

The Intelligence Index that gives Luna its 37 is a composite of ten evaluations — AA-Briefcase, GDPval-AA, AutomationBench and others — each with answers a grader can check. That is what makes it reproducible, and it is why an evaluator with no stake in any model can run it and publish a comparable number for everyone.

PI Knowledge Work was built precisely because the reproducible parts of professional work are not the parts that matter. A diligence memo does not compile. A board deck has no unit tests. A pricing model with a live forecast inside it will be wrong in a way that a multiple-choice grader cannot see. So Paper Instruments wrote rubrics instead, one per task, each criterion drawn as a boundary over acceptable answers rather than as a demand for one. Its report contrasts an overfit criterion that rejects two of three correct responses against a vague one that accepts three wrong ones, then shows the rewritten version matching the boundary — which is a fair demonstration of why this is hard and a fair warning about how much weight a single rubric score should carry.

What the resulting numbers are worth, stated plainly: Ultramarine 1 Flash scores 59.3 on that suite at $0.13 per task. GPT-5.6 Sol, Luna's larger sibling, scores 55.1 at $2.09, and appears across all six Paper Instruments tables — 45.4 on JobBench against Ultramarine's 62.9, 46.8 on FrontierFinance against 53.3, 51.5 on APEX-Agents 1.1 against 57.0. The vendor reports that its suite correlates with the Artificial Analysis Intelligence Index at r = 0.81, which is a serious number and a secondhand one. No third party has run Ultramarine 1 Flash on anything.

The comparison that is measured the same way on both sides

There is exactly one class of measurement in this pairing where neither side's number depends on the other's methodology: token efficiency, measured within a single suite by a single harness.

Artificial Analysis records GPT-5.6 Luna at 150 million output tokens to complete the Intelligence Index, 58% above the 95-million median for its class. That figure sits behind the $0.18 per task — a fast model that writes a great deal is not the cheapest way to buy a finished task, and the verbosity ranking makes the reason visible.

Paper Instruments makes the opposite claim about its own model, and does it in a way that is checkable rather than asserted. Across all six of its benchmarks, Ultramarine 1 Flash emitted 6.5% fewer output tokens than GLM-5.3 Flash, the model it was post-trained from — 30.5 million against 32.7 million, reasoning included — while scoring higher on every one of the six. Both sides of that comparison are the same harness on the same tasks, so it is not vulnerable to the criticism that hangs over everything else here. It is also the direction that does not come for free: post-training runs as often get chattier as they get quieter.

For anyone whose bill is dominated by output tokens, that is the most transferable fact in the release. It does not say Ultramarine 1 Flash is more frugal than GPT-5.6 Luna. It says the shorter-of-the-two-by-base-model relation held, on a suite nobody outside the company has reproduced.

Running the test yourself

GPT-5.6 Luna is served on OrcaRouter at $0.20 input and $1.20 output per million tokens, with the effort level set per request. That is the provider's list price passed through with nothing added per token, which matters for a model whose price has already moved once — the 80% cut OpenAI applied earlier in the year is reflected here without anyone renegotiating a contract. The same endpoint reaches GPT-5.6 Sol, Claude Fable 5, DeepSeek V4 Flash and GLM-5.3 Flash, the last of which is the base Ultramarine 1 Flash was trained from and is served at the identical $0.15 / $0.50 rate card.

Ultramarine 1 Flash is not on our catalogue; the route to it is the vendor's own API and the third-party platforms that resell it. That asymmetry is the practical form of this comparison. One of these two models you can put in a production path today with a routed fallback behind it and a measured p50 in front of you. The other you pilot on the vendor's endpoint and judge on its own output.

Which is the test worth running, and it is cheap: send the same document batch to both, at Luna's default effort and at its maximum, and compare the artifacts and the invoices. The specialist's whole argument is that its output needs less rework. That is measurable in an afternoon and it is the only evidence that would settle a pairing where nobody has published a shared scoreboard.

The position, stated as what is known and what is not

Known, with a source: GPT-5.6 Luna scores 37 on an index built from ten checkable evaluations, at $0.18 per completed task, with a 1,050,000-token window and a first token that can take just over two minutes at the effort level that earns the 37. It is cheap per token, capable for its tier, and — by Artificial Analysis' own measurement — inclined to write 58% more than its peers.

Reported, by its maker, unreproduced: Ultramarine 1 Flash scores 59.3 on a rubric-graded suite of finance, legal, consulting and operations deliverables at $0.13 per task, against $8.19 for Claude Fable 5 at 57.8 and $2.09 for GPT-5.6 Sol at 55.1, and it beats its own base model on all six suites while generating 6.5% fewer tokens.

Unknown: whether any of that survives a document nobody at Paper Instruments wrote, what the model's latency looks like under load, and whether the "Model weights coming soon" line on the vendor's page resolves into an actual release.

Any article that picks a winner from that has invented one. What can be said cleanly is that if you need a verified score before you commit, GPT-5.6 Luna is the only model in this pairing that has one, and if your work is a deliverable rather than an answer, the interesting claim is entirely on the other side and has not been tested by anyone who was not paid to test it.

Comparisons like this one are also where the failure modes show up: the documented AI incidents and agent failures behind models in this class are collected in the OrcaRouter AI Incident Archive , where each entry carries its own sources, severity, confidence grade and disputed flag.