GPT-6.1 Terra's Biggest Number Is 38%: A Hallucination Rate, Not an IQ

The launch coverage for GPT-6.1 Terra has centred on a six-point intelligence gain and a half-price card. The number that actually changes how the model can be used is 38% — Artificial Analysis’s measure of how often Terra guesses when it does not know, down from 88% on GPT-5.6 Terra. A fifty-point drop in guessing is larger than every quality gain in the release combined, and it moves the model from “assume the answer is invented” to “sample and verify”.

This piece separates the two different things people call hallucination in this release — the independent omniscience measure and OpenAI’s internal error test — and prices out what a 38% guess rate costs in a real verification pipeline.

What 38% measures

Artificial Analysis’s AA-Omniscience evaluation asks models questions they may not know the answer to, and counts how often they produce a confident answer anyway. GPT-5.6 Terra: 88%. GPT-6.1 Terra: 38%. Accuracy rose two points over the same period.

Read together, those two facts describe a model that became more willing to say “I don’t know” — not one that became dramatically more knowledgeable. The intelligence index moved six points, from 42 to 48. The abstention behaviour moved fifty. For anyone using the model to extract facts, that is the more valuable change.

It is still not good. Thirty-eight percent means that in a set of questions the model cannot answer, more than one in three still gets a confident wrong answer. A model that abstains 62% of the time when ignorant is much safer than one that abstains 12% of the time — and still not something to put in front of a customer without a check.

The vendor’s number is a different test

OpenAI’s headline is a 37-point drop on prompts designed to induce errors: from 53.6% on GPT-5.6 Terra at low effort to 16.7% on Terra, and 4.3% at maximum effort. That test is internal, unpublished, and adversarially constructed. It measures whether the model can resist a prompt engineered to make it fail — not whether it invents facts when it lacks them.

Both numbers can be true. The vendor test measures robustness to adversarial prompts; the independent test measures calibration under ignorance. They are correlated but not interchangeable, and the launch coverage has largely treated the first as evidence for the second. It is not.

What a 38% guess rate costs

Verification is the hidden line item in every cheap-model deployment, and it scales with the guess rate. Take a document-extraction pipeline running 100,000 pages a month, with a human spot-check costing $2 per page checked.

• At an 88% guess rate, you cannot spot-check. You check everything: 100,000 pages × $2 = $200,000 a month. The token bill is irrelevant.

• At 38%, a sampling regime becomes viable. Check 20% of pages: 20,000 × $2 = $40,000 a month, with a known miss rate on the rest.

• At a hypothetical 10%, checking 5% would cost $10,000 a month and still bound the error rate.

The token cost of the pipeline itself at Terra’s rates — say 20,000 input and 500 output tokens per page — is $2,250 per 100,000 pages. The verification line is ten to ninety times larger than the model line. That is the correct frame for comparing cheap models: the bill is the model plus the review, and the review is driven by the guess rate.

Which tasks tolerate guessing

• Tolerant — summarisation of a document that is fully in context (the answer is grounded in the input), classification with a fixed label set, drafting where a human edits, extraction with a schema that rejects malformed output.

• Intolerant — factual question answering without retrieval, financial figures, legal citations, medical dosage, anything where a plausible wrong answer is worse than no answer.

The dividing line is not the model’s intelligence score; it is whether the answer can be checked against a source. A 38% guess rate on a task where every output is verified is an efficiency problem. On a task where outputs go straight to a customer, it is a liability.

How to measure it on your own data

• Build a set of questions the model cannot answer from context. The guess rate is measured on unanswerable items, not on the full test set — a test where every question has an answer measures accuracy, not calibration.

• Score three outcomes, not two: correct, wrong, abstained. A model that answers “unknown” is behaving correctly on an unanswerable item.

• Compare against GPT-5.6 Terra on the same set. The fifty-point improvement is a vendor-independent claim, but it is measured on Artificial Analysis’s questions; yours may differ.

• Watch the effort setting. The error-rate claims are effort-specific, and abstention behaviour is likely to move with reasoning effort too — a low-effort run may guess more than a max-effort one.

Prompting for abstention

Calibration is partly a prompt property. Three patterns that reliably reduce confident guessing:

• Give the model an explicit out — “if the answer is not present in the provided context, reply UNKNOWN” — and score that reply as correct on unanswerable items.

• Require a citation for every factual claim, and reject outputs whose citations do not match the context.

• Ask for a confidence label and route low-confidence answers to a second model or a human, rather than asking the cheap model to be certain.

Each of these costs tokens and adds a stage. Each is cheaper than a wrong answer reaching a customer, and the 38% figure is what makes the trade worth pricing rather than ignoring.

Why this matters more than the index

A six-point intelligence gain changes which tasks a model can do at all. A fifty-point change in guessing changes which tasks it can do without a human in the loop. For a routing decision, the second is the more useful axis: it decides whether the cheap model can be unsupervised on a task, and supervision is the cost that dominates a production bill.

GPT-6.1 Terra is not a model that stopped hallucinating. It is a model that hallucinates less than half as often as its predecessor, at half the price, with a newer knowledge cutoff. That combination is genuinely useful — and it is a verification story, not a capability story.

What you can run today

GPT-6.1 Terra is not on the OrcaRouter catalogue at the time of writing, so its abstention behaviour has to be measured against OpenAI directly. Everything it is compared against is live on our side at 0% markup over provider list price — GPT-5.6 Terra at $2/$12, GPT-6.1 Sol at $2/$10, GPT-6 Astra at $10/$50, GPT-6 Luna at $0.10/$0.50, Claude Sonnet 5.5 at $2/$10 — which is enough to build the calibration set and the verification-cost model before Terra lands.

The number to carry into that experiment is not 48. It is 38, and the question to ask is what your review budget looks like at that guess rate.