Title card reading 'Astra for Law 6.2' with the line '0.03 material hallucinations per assignment' above three cards showing 20.8% full credit on Harvey LAB-AA, 0.03 material hallucinations, and 64,000 output tokens per assignment, over a footer noting that LAB-AA figures are per Artificial Analysis and also reported by OpenAI.
Guides & Insights

Astra for Law 6.2 Takes First on Harvey's Legal Benchmark. The Hallucination Rate Is the Real News.

Author

Gideon Frost

Date Published

Latest models · 20View all models →
Benchmarks: Artificial Analysis · updated daily
Back to all posts

Ope​nAI shipped a new version of Astra for Law on 17 November 2026 — the legal configuration built on GPT-6.2 Astra, the flagship it launched the day before — and it lands first on the Harvey LAB-AA leaderboard that Artificial Analysis maintains: 20.8% of 120 legal assignments completed to full credit, against 15.0% for GPT-6.2 Astra and 13.1% for Muse Wonder Full, the best non-Ope​nAI entry. The figure worth more of your attention sits two columns to the right. Astra for Law averaged 0.03 material hallucinations per assignment. Claude Opus 5.5, the next-lowest non-Ope​nAI model on that measure, averaged 0.59.

Both numbers describe the same 120-assignment run, which is what makes them worth reading together: a configuration that finishes the whole task slightly more often than anything else tested, and invents a substantive claim roughly twenty times less often than its strongest non-Ope​nAI rival. Here is what changed since September, where the claims hold, where they do not, and what access actually looks like this week.

Same weights, a bigger index, and a new name to search for

Nothing here is a new model in the sense of new weights. Astra for Law is a configuration of GPT-6.2 Astra: the base model plus an expanded legal search index plus instructions for researching a question, weighing the authorities, and writing the answer. That is the same construction Ope​nAI used in September, when the layer sat on GPT-6 Astra. The upgrade is threefold — a newer base model, a larger index, and another two months of tuning against practitioner feedback.

The index is the concrete part. Ope​nAI grew it 34% since September, adding more than 80 million URLs, for a total of over 310 million, and extended coverage to the UK, Australia and Canada. On the U.S. side it claims more than 99.9% of published precedential case law, measured against CourtListener's coverage data. Instructions are invisible and unverifiable; index size is a number you can at least hold someone to.

One practical wrinkle before any of this is useful: the thing has four names. Ope​nAI's post calls it Astra for Law. Artificial Analysis lists the leaderboard entry as Astra for Law 6.2 (Max). The model id — the string you select in ChatGPT once access is enabled — is gpt-6.2-astra-law. Searches for "GPT-6.2 Astra for Law" return the same product. If you are briefing a procurement team, give them the model id and the leaderboard label together, because they will not find each other otherwise.

What 20.8% means, and why it is not the same as "wrong 79% of the time"

LAB-AA is Artificial Analysis's run of Harvey's Legal Agent Benchmark: 120 assignments across 24 areas of law, from drafting deposition outlines to reviewing commercial leases to strategising ownership changes in acquisitions. Scoring is all-or-nothing in a way that matters. An answer earns full credit only if it meets every requirement of the assignment and contains no material hallucination. Miss one sub-requirement, or assert one unsupported claim, and the assignment scores zero.

Read that way, the gap between 20.8% and 15.0% is larger than five percentage points suggests: it is the difference between finishing one in five legal assignments cleanly and one in seven. Ope​nAI's own arithmetic checks out — 20.8 against 15.0 is a 39% relative gain, which its "nearly 40% more often" phrasing describes accurately, and 20.8 against 13.1 is 59%, again as stated.

The dimensions that decide whether this is worth a budget line:

• Full credit on LAB-AA — Astra for Law 20.8% vs GPT-6.2 Astra 15.0% vs Muse Wonder Full 13.1%

• Output tokens per assignment — about 64,000 vs 139,000 for Muse Wonder Full, a 54% reduction at a higher pass rate

• Material hallucinations per assignment — 0.03 vs 0.06 for GPT-6.2 Astra and 0.59 for Claude Opus 5.5

• Minor hallucinations per assignment — 0.06 vs 0.51 for Claude Opus 5.5, about 8.5 times as many

• Cost per LAB-AA assignment at Max effort — $7.33 vs $6.35 for GPT-6.2 Astra and $17.93 for Claude Opus 5.5, per Artificial Analysis

• Zero material hallucinations on tasks that met the all-pass criteria, which is the sub-claim that stops the passes from being bought with invented citations

A scoreboard titled 'Astra for Law — the scoreboard' with six rows reading 'LAB-AA full credit: 20.8%, rank 1', 'Material hallucinations: 0.03 per assignment', 'Output tokens: 64,000 per assignment', 'Vals Legal Research Bench: 53.85%, rank 6 of 99', 'Legal search index: 310M+ URLs' and 'Published API price: none', with a footer noting that LAB-AA figures are per Artificial Analysis and Vals figures per Vals AI's 16 November 2026 evaluation.

The cost line deserves a second look because it cuts against the usual pattern. At the same Max reasoning setting, the legal configuration costs $0.98 more per assignment than its own base model while earning 5.8 points more full credit and generating 54% fewer output tokens than the model it beat. You are not paying for verbosity here; you are paying for a retrieval layer that lets the model stop sooner. For a team that prices legal work by the hour, that is the only arithmetic that matters, and it is the one number in the release that a competitor cannot copy by shipping a bigger model.

The hallucination rate is the claim with the longest half-life

Artificial Analysis counted material hallucinations — claims that could mislead a reader on a substantive point — across the same 120 assignments. Astra for Law averaged 0.03 per assignment, tied with GPT-6 for the lowest rate in the comparison and roughly half the rate of regular GPT-6.2 Astra at 0.06. Claude Opus 5.5 averaged 0.59, which is the "about 20 times" figure Ope​nAI quotes, and the ratio holds: 0.59 divided by 0.03 is 19.7.

One caveat is worth stating plainly rather than buried, because it cuts both ways. At these magnitudes the counts are tiny — 0.03 per assignment across 120 assignments is a handful of material hallucinations in total — so a single bad answer moves the rate materially. The 20-times ratio is a real difference between two measured runs, not a stable physical constant, and it should not be quoted to a client as a guarantee of anything. What it does establish is direction and rough size, and on a task where one invented citation can end a relationship, direction and rough size are worth more than a point estimate.

The provenance also needs separating. Artificial Analysis ran the evaluation, so the numbers are not Ope​nAI's, but Harvey wrote the benchmark and Ope​nAI worked with Harvey on the September rollout. LAB-AA is independent of the vendor, not independent of the benchmark's author. There is no third-party hallucination count on a task set that Ope​nAI had no hand in — that is the gap in the evidence, and it is a bigger gap than the 0.03 headline suggests.

Screenshot of the Artificial Analysis Harvey LAB-AA evaluation page, showing the leaderboard with Astra for Law 6.2 in first place at 20.8% full credit, GPT-6.2 Astra second at 15.0%, GPT-6.2 Astra at High effort third at 13.3%, and Muse Wonder Full fourth at 13.1%, with material hallucination rates of 0.03, 0.06, 0.03 and 0.62 per assignment and costs per task of $7.33, $6.35, $3.86 and unavailable.

Where it is not first

Vals AI's Legal Research Bench, evaluated 16 November 2026, tells a flatter story. Astra for Law sits sixth at 53.85% all-pass on 208 questions that practising lawyers wrote about U.S. federal and state law, behind Gemini 4 Carbon at 55.77% and a three-way tie at 55.29% between Muse Spark 1.3 Max, Claude Opus 5 and Claude Fable 5.1.

Sixth by 1.9 points, but first on the economics of the top group:

• All-pass accuracy — Astra for Law 53.85% vs Gemini 4 Carbon 55.77% and Gemini 4 Argon 54.81%

• Cost per test — $2.75 vs $8.74 for Gemini 4 Carbon, $6.76 for Claude Opus 5 and $23.06 for Claude Fable 5.1

• Tokens per test — 77,740 vs 5.36 million for Gemini 4 Carbon and 3.13 million for Claude Opus 5

• Duration per test — 16 minutes 45 seconds vs 51 minutes 53 seconds for Gemini 4 Carbon

That last pair is the part that does not fit the headline. It is not just cheaper per answer; it is an order of magnitude leaner on tokens than the models above it, which is the same signal the LAB-AA token count gave.

Ope​nAI's own framing of this benchmark is also worth checking line by line. It reports 112 of 208 questions at full credit, a gain of about eight percentage points from roughly 46% in September, and 90.9% of available points once partial credit is counted. The 112 and the 90.9% reconcile with the public board. The 46% starting point does not appear on it, so the "eight points in under two months" is a comparison Ope​nAI is making against its own September run and cannot be re-derived from what Vals publishes today.

The most useful figure on that board is one Ope​nAI did not lead with. GPT-6.2 Astra, the same base model without the legal layer, scores 42.79% and sits 31st; GPT-6 Astra sits 42nd at 39.42%. So on a benchmark neither Ope​nAI nor Harvey controls, the configuration is worth about eleven points over its own base. That is the cleanest independent estimate of what the legal index and instructions actually add, and it is considerably larger than the 20.8-versus-15.0 gap on LAB-AA implies.

Screenshot of the Vals AI Legal Research Bench leaderboard dated 16 November 2026, showing Gemini 4 Carbon first at 55.77%, Muse Spark 1.3 Max and Claude Opus 5 tied at 55.29%, Gemini 4 Argon fifth at 54.81% and Astra for Law sixth at 53.85% with a cost of $2.75 per test, alongside GPT-6.2 Astra at 42.79% in 31st place.

What it costs, and the number Ope​nAI has not published

There is no price for the legal configuration. That is not an omission in the launch post; it is the launch post. Access is a request, not a checkout: every ChatGPT Enterprise customer can now ask for Astra for Law through the Trusted Access Program in the admin console, teams that already have it select gpt-6.2-astra-law in ChatGPT, and API access goes through a sales contact form rather than a self-serve key.

For budgeting, the base model is the only published rate: GPT-6.2 Astra lists at $8 per million input tokens and $40 per million output, with cached input at $0.40, a 1.05 million-token context window, 128,000 maximum output tokens and a knowledge cutoff of 18 May 2026. Artificial Analysis's record for the law configuration carries the same $8 and $40 — but that is the base model's rate card carried across, and it should be treated as that lab's working assumption rather than a vendor quote. The derived figure is more honest about usage: $7.33 per LAB-AA assignment at Max effort, which reflects how many tokens the configuration actually spends rather than what a token nominally costs.

If you are building the retrieval layer yourself instead of buying the configuration, the base models are reachable on one key through OrcaRouter, which carries GPT-6 Astra and the surrounding Ope​nAI line inside a 215-model catalogue at provider list price with 0% markup and automatic failover when a provider stalls mid-run. Astra for Law is not in that catalogue and there is no public endpoint that reaches it — nothing in this route is a shortcut to the configuration. What the routing layer is good for is the part you can control: a research step that must not silently drop a citation is a reasonable candidate for a second model checking the first, and that composition is a routing-DSL call rather than two vendor integrations.

Ope​nAI published one comparison that answers the question its own headline skips. On Harvey's own Legal Agent Benchmark — a different test from LAB-AA, run by Harvey rather than Artificial Analysis — GPT-6 Astra met every requirement on 4.6% of assignments, GPT-6.2 Astra tripled that to 14.6%, and the legal configuration took it to 16.2%.

Read the sequence, not the last number. Two-thirds of the way from 4.6% to 16.2% is base-model progress: the jump from GPT-6 Astra to GPT-6.2 Astra, which any customer of the base model gets without requesting access to anything. The configuration contributes the last 1.6 points, an 11% relative improvement over the base. That is a real gain and it is the gain you are being sold, but it is a smaller share of the total than launch coverage implies — and it is the part most likely to be replicated by other labs' own retrieval layers within a quarter.

Who should request access, and who should wait

If you are inside a ChatGPT Enterprise tenant and your lawyers already do research in it, request access this week. The cost of the request is a form, and the first useful thing you can do with it is not run Harvey's benchmark but your own: twenty assignments from your own practice, scored all-or-nothing with a hallucination veto, which is a rubric you can build in an afternoon and which will tell you more about your work than 20.8% on someone else's does.

If you are outside Enterprise and outside the API programme, there is nothing to buy yet — no rate card, no key, no partner availability. Waiting costs you nothing that a quarter of access would not fix, and the absence of a published price is itself information: the vendor has not decided what the legal layer is worth independently of the base model.

If you are building, the transferable finding is not the leaderboard position. It is that the best-scoring configuration on LAB-AA used 54% fewer output tokens than the model beneath it in the ranking. Per-task economics, not per-token price, is where legal AI gets won, and a retrieval layer that lets the model stop talking sooner is cheaper than a smarter model that keeps going.

Three things to watch next. A published price for the configuration, which will say whether Ope​nAI sees this as a premium product or a feature bundled into Enterprise. API availability beyond the request form, which determines whether anyone can reproduce the numbers independently. And a non-U.S. hallucination count: the index now covers the UK, Australia and Canada, but every published figure measures U.S. law, and the failure mode that hurts most in other jurisdictions — citing the right provision while misreading it — is the one no index expansion fixes on its own.

Compared in this article2

Detected from this article · Benchmarks: Artificial Analysis · updated daily