SubQ Snap 1.1 Ties for First on the Decision Index by Learning to Abstain

The newest model on top of the Artificial Analysis Decision Index v1.0 is not the one that gets the most answers right. SubQ Snap 1.1, released by Subquadratic on October 22, 2026, scored 65 and tied d1-ultra and Elephant 9B for first place. Jev 1.14 sits at 57. Read the fine print underneath the index and the reason for the tie is not raw accuracy at all: Artificial Analysis's write-up puts Snap 1.1's accuracy at 20%, against 32% for d1-ultra and 12% for Elephant 9B. What Snap 1.1 does better than either is refuse to answer.

That is a real engineering result, not a benchmark trick, but it changes who should care. A decision model is not a chat model. It takes a document and a fixed set of allowed answers — route to this queue or that one, call this tool or decline, yes or no or escalate — and returns a choice with a calibrated probability attached. Subquadratic's pitch since the preview shipped ten days earlier has been that this workload deserves its own architecture rather than a general LLM burning output tokens to produce the word "refund". Version 1.1 is the first release where the calibration claim is specific enough to check.

What actually changed between Snap 1.0 and 1.1

This is a post-training release, not a new model. The Subquadratic Sparse Attention backbone, the 384K-token window, and the text-plus-image input path are all unchanged from Snap 1.0, which went live on October 12 at $0.05 per million input tokens. What moved is how the model behaves when it is uncertain: the vendor says 1.1 adds explicit uncertainty calibration and distils multi-step reasoning into a single forward pass, rather than generating a chain of thought and then committing to an answer.

The numbers Subquadratic reports for that change are unusually specific, which is both good and worth scrutinising. Expected calibration error on held-out probability benchmarks fell from 0.108 to 0.014. Confident errors on unanswerable questions dropped from 294 out of 600 to 57 out of 600 — the 81% reduction the launch coverage keeps quoting. Overconfident hallucinations on out-of-distribution inputs fell from 99.3% to 12.5%. Agreement with a Bayes-optimal decision rule rose from 83% to 90%. On the vendor's 16-benchmark enterprise suite, 1.1 averages 7.2 points above Jev 1.14 with a 95% confidence interval of 6.5 to 7.8, winning 12, drawing 3 and losing 1, and averages 5.5 points above Snap 1.0.

All of that is vendor-reported. Nobody outside Subquadratic has published a run of 1.1 yet; the independent repos that exist tested the preview model, and they are discussed further down.

How a model with 20% accuracy ties for first

The Decision Index does not score accuracy in isolation. It scores the outcome of a decision under uncertainty, which means an abstention on a question the model cannot answer is worth more than a confident guess — and worth much more than a confident wrong guess. Artificial Analysis reports Snap 1.1's non-hallucination rate, meaning how often it said "I don't know" on questions it could not answer correctly, at 86%. Elephant 9B is higher at 93% and d1-ultra is marginally lower at 85%. Snap 1.1's 20% accuracy is the middle of that trio, and its index score is identical to both.

Whether that is the right trade depends entirely on what a wrong answer costs you downstream. In a support-ticket router, a confident misroute sends a billing complaint to the engineering queue and a customer waits two days; an abstention sends it to a human triage step that costs thirty seconds. In a bulk content classification job with no human in the loop, abstention is just an unlabelled row and the accuracy number is the one that matters. The index is telling you Snap 1.1 is built for the first case. It is not telling you it is the most accurate classifier you can buy, because it isn't.

The price cut, and the denominator nobody quotes

The other change in 1.1 is the price: $0.04 per million input tokens, down 20% from the preview's $0.05, with output tokens free. That second half is the part that reshapes the bill. A classifier built on a generative model pays for the answer — usually a short one, but reasoning models can spend hundreds of tokens deciding what to emit. Snap has no output to pay for, so a call that reads 100K tokens costs about four-tenths of a cent and nothing else.

Artificial Analysis frames the same figure per task rather than per token: roughly $30 per million tasks for Snap 1.1, against $32 for d1-ultra and $8.4 for Elephant 9B. That per-task number is the useful one, and it is also a reminder that the per-token list price is a poor way to compare a decision model to anything else. The bill scales with input length, so a 4K-token routing call and a 300K-token contract review land in completely different cost brackets even though the rate is identical.

What the vendor's numbers do not settle

Two things are worth holding apart from the launch post. The first is skepticism. Community coverage of Subquadratic's releases — including the BlockBeats reporting that circulated as "AI jack-of-all-trades" — has questioned how much of the claim set is independently reproducible, and the model has no open weights, so there is no checkpoint for an outsider to run. The second is that the independent tests that do exist were run against the preview, not 1.1. A reproducible comparison by jcarlosrodicio found Snap slightly behind Jev below 30K tokens (93.9% against 94.7%) and well behind on 360K-token logs (58.8% against Jev's 77.1% when Jev was chunked). A separate software-engineering benchmark, snap-vs-jev-swe, found Snap at 87.5% on SWE-bench Verified file localisation against a 10.6% random baseline, and measured answer flips on 5.0% of items when the option order was reversed.

Those results are not contradicted by 1.1 — the calibration work is exactly the kind of change that would move them — but they are not evidence for it either. There is also a small internal inconsistency in the published figures: the launch coverage cites a When2Call score of 96.2, while Subquadratic's own chance-corrected multiple-choice variant of the same benchmark reads 91.0. Both can be true of different task formats; a buyer comparing the 96.2 to a competitor's number should know which variant they are holding.

Where it breaks

The loss column of the vendor's own 16-benchmark suite is instructive. PhishNChips, a phishing-detection set, comes in 22 points behind Jev 1.14, and Subquadratic attributes that to threshold calibration rather than ranking quality — ROC-AUC is roughly level at 0.91 against 0.92. That is a fixable problem for a team willing to tune a threshold, and an unfixable one for a team that wanted a drop-in. The knowledge-and-reasoning subarea is the only one of five where Jev leads outright, 55.1 to 50.9. Snap 1.1 is strongest where the answer set is explicit and the evidence is in the document: tools and automation (85.3 against 74.6), retrieval and classification (67.0 against 63.1), language understanding (71.6 against 63.3).

Who should move, and who should wait

Move if your decision is a routing, tool-selection, escalation or relevance call made over long documents, and a confident wrong answer costs more than an escalation. The 384K window is the operational reason: a single pass over a full contract, ticket thread or log bundle, with no chunking layer to build and no retrieval pipeline to keep in sync. Teams already running the preview have the clearest case, since the price fell and the calibration improved without an API change.

Wait if you need generated text, if 20% accuracy on the benchmark's question mix is below your floor, or if you need sub-100ms latency in a hot path — Subquadratic publishes no latency figure for 1.1, and the third-party measurement of the preview put an 8K-token call at about 0.75 seconds. Wait too if you need to self-host: there are no weights.

One practical note on where this sits in a stack. SubQ Snap 1.1 is served through Subquadratic's own API and console; it is not in OrcaRouter's catalogue, and we won't pretend otherwise. What we do carry is the layer behind the decision: when Snap resolves a call to "use a model", that model is one of 200+ reachable through a single key at OrcaRouter, at provider list price with 0% markup and automatic failover between providers. The decision layer and the execution layer fail independently, and it is worth not having both behind the same vendor.

The open question is not whether 1.1's calibration numbers are real — they are specific enough that they will be checked — but whether they survive a different harness. Prompt wording moved accuracy by 11 to 14 points in the public Snap benchmarks, more than the gap between any two models on the index. A calibration improvement that only holds under the vendor's own prompt template would be a much smaller thing than the launch suggests.