A generated title card headed ‘First place, last place, same board’, subtitled ‘Liquid AI d1.2-ultra - Decision Index v2.1, November 16 2026’. Two rounded cards sit side by side: the left reads ‘#1 of 24’ above ‘Decision Index 75’, the right reads ‘#24 of 24’ above ‘HLE accuracy 2.36%’; beneath them a caption line reads ‘one model, six evaluations’, and three outlined chips read ‘measured by Artificial Analysis’, ‘66k context’ and ‘$0.042 per 1M input tokens’. The OrcaRouter mark is composited bottom right.
Guides & Insights

Liquid AI d1.2-ultra Tops the Decision Index at 75 — and Finishes Last on One of Its Six Tests

Author

jinhao.song

Date Published

Latest models · 20View all models →
Benchmarks: Artificial Analysis · updated daily
Back to all posts

Liquid AI d1.2-ultra is the first model in this category to be given a first place by somebody other than the company that built it. Artificial Analysis scored it at 75 on Decision Index v2.1, printed that number on the model's own board page, and ranked it first of 24 decision models — 2.2 points clear of Nace.AI's Drex Max v1.1 at 73. Liquid AI d1.2, the smaller half of the same release, lands ninth at 69. Both shipped on November 16, 2026, which makes this the first Liquid AI launch where the leaderboard claim arrived with an independent board already carrying the model, rather than a vendor number pointing at a board that predated it.

The same board also ranks d1.2-ultra last of those 24 models on HLE-MultiChoice, at 2.36% raw accuracy. Both statements are true at once, they are measured by the same organisation on the same instrument on the same day, and the space between them is the most useful thing in the release.

Why this leaderboard claim is a different kind of claim

In October, Liquid AI announced d1-ultra with a first-place claim measured against Decision Index v1.0, a board published two days before that model existed. Nobody could check it, because the model was not on the board. That is not what happened here. Artificial Analysis ran d1.2-ultra itself, on its own hardware, through its own harness, and the result is a ranked row with six component scores and a calibration curve attached. When Liquid says "the highest on the index and two points ahead of the previous best decision model," the index and the previous best are both public and both agree.

Version 2.1 is a weighted average of six evaluations, and the weights are not evenly spread. AA-Paperclip carries 30%: 1,500 private documents — resumes, receipts, invoices, research papers — each carrying five decisions and ten facts, scored as one document that only counts if all fifteen answers come back right. EvalSafe-O*NET-AA, AA-Omniscience-Decision v1.1, HLE-MultiChoice v1.1 and AA-Symmetry v1.1 each carry 15%. When2Call, NVIDIA's tool-calling set, carries 10%. Each question is scored as 50 × (1 + c − (p − c)²), where c is whether the pick was right and p is the probability the model put on it, which aggregates to 50 × (1 + accuracy − Brier score). A confident error costs more than a hesitant one, and a model that says 0.6 and is right scores better than one that says 0.99 and is right.

Where the 75 comes from

A generated single-column scoreboard titled ‘Liquid AI d1.2-ultra - the scoreboard’, with six rows: Decision Index 75, first of 24; AA-Paperclip 75.8 with 67.7% of documents all-correct; AA-Symmetry 86.3, second of 24; When2Call 96.5, first of 24; HLE-MultiChoice 2.36% accuracy, last of 24; and Cost $133 per 1M tasks at $0.042 per 1M input tokens. The footer reads ‘All figures measured independently by Artificial Analysis, Decision Index v2.1; Liquid AI's own task accuracies are vendor-reported.’

Pulled apart, the first place is carried by two evaluations and defended by a third. The rest are unremarkable or worse, which is worth knowing before anyone treats a composite score as a description of the model.

• AA-Paperclip, 30% of the index — d1.2-ultra scores 75.81 on the decision score with 67.7% of documents fully correct, both first of 24. Drex Max v1.1 is second at 72.82 with 63.0%, SubQ Snap 1.2 third at 71.58. The model it replaces, Liquid AI d1-ultra, sat at 64.69 and 51.3%.

• When2Call, 10% — d1.2-ultra is first at 96.49, ahead of SubQ Snap 1.2 at 96.10 and d1.2 at 96.09. Tool calling was already the field's strongest suit; the spread here is a point and a half across the top ten.

• AA-Symmetry, 15% — second at 86.34, behind Mammoth 27B's 88.25. On the stricter all-pass measure the picture changes: Mammoth holds the same answer across paraphrases on 68.3% of questions, Elephant 9B on 65.8%, and d1.2-ultra on 56.7%. Second place on the score is not second place on consistency.

• EvalSafe-O*NET-AA, 15% — third at 89.53, behind Jev 1.14 at 91.14. This is the one component where a competitor's model still leads, and it is not close enough to call noise.

• AA-Omniscience-Decision v1.1, 15% — 60.80, which is mid-field, and below the 64.39 that d1-ultra scored.

• HLE-MultiChoice v1.1, 15% — 46.36 on the decision score, but 2.36% raw accuracy, the lowest of all 24. That gap is not a contradiction; it is what a well-calibrated model looks like when it declines almost everything. Its non-hallucination rate on the same set is 76.1%, meaning three times in four it correctly said it did not know rather than guessing.

The headline metric underneath all six, AA's separate accuracy index score, puts d1.2-ultra first at 61.6, ahead of GLiDE v1.2 at 59.4 and Drex Max v1.1 at 58.2. Its Brier score of 0.150 is the best of the 24. The model is not winning because it is calibrated and nothing else; it is winning on documents and calibration together, which is the combination the index was built to reward.

The score that goes the other way

There is a version of this release that reads as a clean sweep, and the HLE-MultiChoice row is what refutes it. Two and a third percent raw accuracy on Humanity's Last Exam multiple choice is not a rounding error, and it is not explained away by the decline option: the model has the option to abstain, and it takes it, and the abstention is scored as honesty rather than as knowledge. Decider v1.3 answers 15.32% of the same questions correctly. Drex Max v1.1 answers 8.25%. Liquid AI d1.2, the cheap model in the same release, answers 5.30%.

That last comparison is the strange one. On the knowledge-heavy evaluations the smaller, cheaper model is not the weaker one. d1.2 gets 30.0% of AA-Omniscience-Decision questions right against d1.2-ultra's 23.6%, and 5.30% on HLE against 2.36%. What d1.2-ultra buys with its extra two cents per million tokens is not knowledge; it is document handling, symmetry and calibration — the operational half of the job. If your workload is extraction from a scanned invoice, that trade is obviously correct. If it is answering hard questions about the world, the cheaper model in the same family is measurably the better one, and nothing in the launch post says so.

The billing change that is not in the headline

The most consequential sentence in Liquid's post is a billing rule, not a benchmark. Under d1-ultra, every question was billed as its own prompt including all attached images, so asking ten questions about one photograph paid for the photograph ten times. Under d1.2 and d1.2-ultra the request bills its text and pictures once, and each additional question adds only its own text.

Take a 2,000-token document with one 1024×1024 image, which costs 1,536 tokens at the stated rate of 1.5 tokens per 32×32-pixel patch, and ten short questions at roughly 30 tokens each. On the old rule that request carries about 35,700 tokens. On the new one it carries about 3,840. At $0.042 per million input tokens that is roughly $0.0015 per request against $0.00016 — a little over nine times cheaper for identical work at an identical sticker price. At 100,000 requests a month the difference is about $150 against $16. Both figures are arithmetic on the vendor's own billing description, not measurements.

The second spec that moved is the context window. Artificial Analysis lists 66k tokens for both d1.2 models, against 33k for d1-ultra. Liquid's post does not state a window at all, so the 66k figure is the board's reading rather than a vendor specification, and it is still double what the previous generation carried. For a model whose entire job is to be handed state, that is a bigger practical change than four index points.

Five new answer types, and audio

The capability additions are additive rather than a rework: the same endpoint, the same request shape, five new kinds of answer and a new input modality. A question can now ask for a value read off the page — a number, a date, a name — returned with the characters it was read from, or "not found" if it is absent. It can ask for every span of a given kind with positions, which is the PII case. It can ask for a box around one object or a box around every instance plus a count. It can ask for an estimate with a range attached. And it can ask for every applicable option, each with its own probability.

Audio is the other addition, working in both directions: the recording can be what the question is about, or it can be the question itself. Liquid reports that on held-out recordings both models answer 95% to 97% of questions about which sound they hear, from ESC-50, or which language is spoken, from FLEURS across 43 languages, with three to six options each and close confusions such as Spanish and Portuguese in the mix.

The accuracy figures for the five new tasks come from internal test sets built from held-out splits of public datasets, and the vendor says so plainly, because no public benchmark covers these tasks yet. Read them as a vendor's own evaluation: 98% and 96% for reading a value from a receipt on CORD and SROIE, 94%/93% precision and recall against 92%/88% for PII spans on Nemotron PII, 94% for both on object location across the RefCOCO family, 93% and 92% on counting within one across COCO, and 87% against 90% on picking every object present — the one row where the cheap model wins.

Checking the money claim

A headless-browser screenshot of the Artificial Analysis Decision Index v2.1 page, captured 17 November 2026 in English, headed ‘Artificial Analysis Decision Index v2.1’ and described as a composite index testing decision models across document decisions and extraction, tool calling, knowledge, reasoning and invariance, with a note that version 2.1 combines six evaluations starting with AA-Paperclip's 1,500 resumes, receipts, invoices and research papers.

Liquid's pricing sentence is unusually checkable: "no model scores higher than d1.2 for less money, and no model scores higher than d1.2-ultra at any price." Both halves survive contact with the board. Nothing on the 24-model index scores above 74.85 at all, so the second half is trivially true. For the first, d1.2 sits at 68.69 and $63 per million tasks; every model scoring above it costs more, with the nearest cheap challenger being Decider v1.3 at 70.41 and $68. The claim holds, and it holds on somebody else's numbers rather than Liquid's, which is rarer than it should be.

The cost figures themselves come from Artificial Analysis, which runs each evaluation at current list prices and averages: $133 per million tasks for d1.2-ultra, eleventh of 24 against a class median of $135, and $63 for d1.2, third cheapest. Time per task is 218 ms for the big model and 150 ms for the small one, against a class median of 209 ms — the flagship is slightly slower than average, the cheap model comfortably faster, and both are an order of magnitude quicker than the generative models that appear on the same board at eleven and thirty-two seconds per task.

Where you can call it, and what to run it against

Both models are live today on Liquid's own API at api.liquid.ai/decisions/v1/systemone, called as d1.2-ultra and d1.2, with keys from the Liquid console, and both are on the Liquid playground. Liquid's post says additional third-party platforms are coming; as of this writing the vendor's own API is the route that exists. Neither model is on our catalogue, and this article is not an availability claim for either.

What is on our catalogue is Jev 1.14, the model that leads the one component d1.2-ultra does not — EvalSafe-O*NET-AA, at 91.14 against 89.53 — served over the same systemone endpoint shape, text in and structured JSON out, at $0.042 per million input tokens. That coincidence is useful: a team evaluating this release can put the new model and the incumbent on the same states without a second contract or a second client library, because one API for 200+ models already speaks both shapes. List prices are passed through at 0% markup, so if Liquid moves the $0.042 the number here moves with it rather than after it, and automatic failover means a model that is one day old can take a slice of a real workload without being trusted with all of it.

A headless-browser screenshot of the OrcaRouter model page for typesafe/jev-1.14, captured 17 November 2026 in English, showing the /v1/systemone endpoint, Zero Data Retention availability, an input and output list price of $0.04, and measured latencies of 182 ms and 372 ms.

The recommendation that follows from the data rather than from the pricing page: shadow-run it. d1.2-ultra is first on documents and first on calibration, which is a genuinely strong pair for extraction pipelines, and it is last on hard knowledge questions, which is a real limitation rather than a footnote. Send it the states your pipeline actually sees, compare the probabilities rather than the top-1 picks, and watch the cases where it answers 0.8 and is wrong, because that tail is where the model's October predecessor scored badly and where the improvement either shows up or does not.

The open question is not whether 75 is real — Artificial Analysis measured it and published the components. It is whether a 30%-weighted document evaluation, on 1,500 private documents, generalises to your documents. That is a question only your own labelled sample can answer, and it is worth answering before the billing change tempts you into asking ten questions per document instead of one.