A generated title card headlined 'GPT-6.2 Astra Tops Blueprint-Bench 2' with the subtitle 'the human line is twelve apartments' and a caption reading '0.614 against a 0.586 human baseline'. Two rounded cards sit below: the left labelled 'GPT-6.2 Astra' with the rows 'Blueprint-Bench 2: 0.614', 'rank 1 on the board' and '3D scene reconstruction'; the right labelled 'human baseline' with '0.586', 'measured on 12 apartments' and 'of 50 in the benchmark'. The OrcaRouter logo is composited in the bottom-right corner.
Guides & Insights

GPT-6.2 Astra Tops Blueprint-Bench 2: the Human Line Is Twelve Apartments

Author

jinhao.song

Date Published

Latest models · 20View all models →
Benchmarks: Artificial Analysis · updated daily
Back to all posts

GPT-6.2 Astra scores 0.614 on Blueprint-Bench 2. The human baseline on the same board is 0.586. That is a real lead, and it is also 0.028 of a point measured against a human number that Andon Labs computed from twelve apartments. The result went up on Andon's evaluation page on 20 November 2026 and was amplified the same morning by Greg Brockman, who put it as GPT-6.2 Astra "exceeding human baseline on physical 3D understanding benchmark." Both halves of that sentence need a little air let out of them, and this article is about which parts survive. GPT-6.2 Astra, GPT-6.2 Sol and Gemini 4 Carbon occupy the top four slots on Andon's board, which matters more for a reader deciding what to call than the headline rank does.

The benchmark itself is not new. Blueprint-Bench 2 has been on Andon Labs' site since May 2026 and has been collecting models for six months; what changed on 20 November is that a model finally cleared the human line by more than its own error bar. So this is not a benchmark launch and it is not a model launch — O​penAI shipped GPT-6.2 Astra on 17 November 2026, three days before the score appeared. It is a result landing on an existing board, and the interesting question is not whether the number is real. It is what the number is measuring, since the phrase "physical 3D understanding" is doing more work in the tweet than the eval does in its own methodology.

What Andon Labs actually measured

Blueprint-Bench 2 gives an agent fifty apartments, one after another. For each apartment it looks at roughly twenty interior photographs and has to draw a 2D floor plan: rooms, which rooms connect to which, and their relative sizes. Agents carry a persistent notepad between apartments, so a model can write down what it has learned about typical layouts — where a hallway usually sits, what a doorframe in a corner implies — and use it on the next flat. That notepad is the part of the design that makes this an agent eval rather than a vision test, and it is why the scores separate frontier models from mid-tier ones more cleanly than a single-image benchmark would.

Scoring compares the model's connectivity graph against ground truth, weighted across six sub-metrics: Jaccard similarity on room-to-room connections at 50%, degree similarity at 20%, graph density at 10%, room count at 10%, door count at 5% and orientation at 5%. Plans are matched under D4 symmetry, so a plan that is rotated or mirrored still scores, and the composite is normalised so a random baseline maps to 0 and a perfect plan maps to 1. The 50% weight on Jaccard is the whole benchmark in one line: Andon's own writeup says every model gets room count roughly right, around 90%, and that the discriminator is whether the model can work out which rooms open into which. Counting rooms from photographs is easy. Reconstructing the topology from a walk-through is not.

Two consequences follow from that design. First, the output is a 2D plan, not a 3D model — the spatial reasoning is three-dimensional in the sense that the agent has to infer a floor's layout from a sequence of overlapping views, but nothing in the scoring rewards a correct reconstruction of ceiling height, volume or furniture. Second, the score is a graph-similarity number, not an accuracy percentage. A 0.614 does not mean the model got 61% of the apartments right; it means its connectivity graphs sit 61.4% of the way from random to perfect under that weighting.

The 0.028 that separates a model from a person

Here is the top of the board as it stood on 20 November 2026, with the standard-error bars Andon draws on its chart read off the published figure:

• GPT-6.2 Astra — 0.614, rank 1, error bar roughly 0.601 to 0.627

• Gemini 4 Carbon — 0.586, tied rank 2, error bar roughly 0.571 to 0.601

• Human baseline — 0.586, tied rank 2, tested on 12 apartments, no interval published

• GPT-6.2 Sol — 0.570, rank 4, error bar roughly 0.558 to 0.584

• Claude Fable 5.2 — 0.566, rank 5

• Gemini 4 Argon — 0.544, rank 6

• Claude Opus 5.5 — 0.512, rank 7

• GPT-6 Astra — 0.497, rank 8

A screenshot of Andon Labs' Blueprint-Bench 2 evaluation page. The top half explains that agents convert roughly twenty interior photographs of each apartment into a 2D floor plan across fifty apartments, with the normalisation note that random maps to 0 and perfect to 1 and that error bars are standard error, above a connectivity-similarity chart. The leaderboard below lists GPT-6.2 Astra at 0.614 in first place, then Gemini 4 Carbon and a Human baseline both at 0.586, GPT-6.2 Sol at 0.570 and Claude Fable 5.2 at 0.566, continuing down through GPT-6 Astra at 0.497 to Claude Haiku 5.5 at 0.330.

Three things are worth pulling out of that list. The first is the size of the margin, which is 0.028 of a point on a scale where random is 0 and perfect is 1. The second is that Astra's error bar does not reach down to 0.586 — the bottom of its band is 0.601 — so the lead over the human figure is not inside the model's own noise. The third is that the human number carries no uncertainty band at all, because it was computed on a dozen apartments out of a fifty-apartment task. A human score from twelve flats and a model score from fifty are not the same measurement, and Andon is upfront about it: the leaderboard footnote says "Human baseline tested on subset of 12 apartments only." Treat 0.586 as an estimate of where a person lands, not as a line with a known width.

The other thing the error bars show is how thin the top of the board is. Astra's lower whisker and Gemini 4 Carbon's upper whisker meet at exactly 0.601. On this benchmark, first place and tied-second are one whisker-touch apart, which is a different claim from "better than any other AI" and a more defensible one. Andon's own chart caption splits the difference in a way that is worth noticing: it labels the dashed human line "Human (0.59)" and then writes that GPT-6.2 Astra and Gemini 4 Carbon "now ties it," even though the table puts Astra 0.028 above it and Carbon exactly on it. The table is the more careful artifact. The caption is the one that got quoted.

Two benchmarks, one direction

One board clearing a human line could be a scoring artifact. The reason to take the result seriously is that a second, independently built 3D evaluation moved the same way in the same week. Artificial Analysis runs AA-3D, which is a different task entirely: models work as agents in an offline sandbox through Stirrup and turn a written brief into an interactive three.js scene, and the ranking comes from blind pairwise votes by a recruited human panel — over 20,000 votes on the current board — rather than from a graph comparison.

• GPT-6.2 Astra (Max) — 1,318 Elo, 95% interval 1,286 to 1,349, rank 1

• Claude Fable 5.2 (Max) — 1,271 Elo, rank 2

• GPT-6.2 Sol (Max) — 1,259 Elo, rank 3

• Claude Sonnet 5.5 (Max) — 1,204 Elo, rank 4

• Claude Opus 5.5 (Max) — 1,202 Elo, rank 5

• GPT-6 Astra (Max) — 1,168 Elo, 95% interval 1,141 to 1,196, rank 6

The two boards disagree about the middle of the field — Claude Fable 5.2 is 5th on Andon's and 2nd on Artificial Analysis's — and they agree completely about the top. Astra leads, Sol is in the top three, and the previous generation sits roughly 150 Elo back. O​penAI's own launch post made the AA-3D claim on 17 November with the same three numbers, and the independent board now shows them unchanged, which is the rare case where a vendor figure and a third-party figure are the same figure. That is not true of the Blueprint-Bench number: O​penAI said nothing about floorplans, and the score arrived from Andon alone.

A screenshot of the AA-3D Leaderboard from Artificial Analysis, with columns for rank, score range, model and Elo. GPT-6.2 Astra (Max) sits at rank 1 with 1318, above Claude Fable 5.2 (Max), GPT-6.2 Sol (Max), Claude Sonnet 5.5 (Max) and Claude Opus 5.5 (Max), with GPT-6 Astra (Max) further down at 1168.

The convergence is the argument. A graph-similarity benchmark scored by code, and an Elo benchmark scored by blind human preference, are about as far apart as two evaluations of "spatial ability" can be, and they rank the same three models in the same order. What they do not establish is that either one measures something a robotics team would call spatial intelligence, which brings us to the most interesting line on Andon's table.

Why a robotics model scored zero

Gemini Robotics-ER 1.6 scores 0.000 on Blueprint-Bench 2 — at or below the random baseline, flagged by Andon with a double asterisk. It is the most useful number on the page. A model built specifically for spatial and embodied reasoning, the kind of thing you would reach for if the task were "understand a physical space," produces connectivity graphs no better than chance. Andon flags it themselves: the spatial specialisation does not transfer to floorplan generation.

Four other models also sit at the random baseline, and the shape of that cluster tells you what the benchmark rewards. What it rewards is a specific composite skill — read a sequence of photographs, maintain a consistent hypothesis about a floor, and emit a strictly formatted plan that a parser can score. A model can be excellent at reasoning about physical space and still fail this, because the failure mode is usually the output format or the long-horizon consistency, not the spatial inference. Which means "exceeds the human baseline on physical 3D understanding" is a fair summary of the score and a slightly generous summary of the skill. The honest version is narrower: it reconstructs the connectivity of an apartment from photographs more accurately than a twelve-apartment human sample did, on a benchmark that scores graph structure and nothing else.

For anyone whose actual job involves floorplans — property capture, insurance documentation, facilities work, the digitisation of paper plans — that narrower claim is still the useful one, because graph structure is exactly what those workflows need and exactly what a PDF-to-CAD pipeline gets wrong. For anyone reading it as a statement about robots navigating rooms, the Robotics-ER row is the correction.

What it costs to buy that lead

GPT-6.2 Astra lists at $8 per million input tokens, $0.40 cached and $40 per million output, which is 20% below GPT-6 Astra on both directions of the meter. The model that ties the human line, Gemini 4 Carbon, lists at $2 and $10 — a quarter of Astra's input price and a quarter of its output price. GPT-6.2 Sol, fourth on Andon's board at 0.570 and third on AA-3D at 1,259 Elo, keeps GPT-6.1 Sol's token prices rather than Astra's.

• Blueprint-Bench 2 — Gemini 4 Carbon 0.586 vs GPT-6.2 Astra 0.614

• Input price per 1M — Carbon $2.00 vs Astra $8.00

• Output price per 1M — Carbon $10.00 vs Astra $40.00

• AA-3D Elo — Carbon not on the board vs Astra 1,318, rank 1

• Where each sits — Carbon ties the human line vs Astra clears it by 0.028

A generated single-column scoreboard titled 'GPT-6.2 Astra — the scoreboard' with six rows reading 'Blueprint-Bench 2: 0.614, rank 1', 'Human baseline: 0.586 on 12 apartments', 'Error bar: 0.601 to 0.627', 'AA-3D Elo: 1,318, rank 1', 'AA-3D vs GPT-6 Astra: +150 Elo' and 'Price per 1M: $8 in / $40 out', over a footer reading 'Blueprint-Bench 2 figures per Andon Labs; AA-3D per Artificial Analysis; pricing per OpenAI.' The OrcaRouter logo is composited in the bottom-right corner.

Read that as a pricing question and the answer is uncomfortable for the flagship. On the one benchmark where both models appear, a fifth of the price buys you a tie with the human baseline and a whisker's distance from first place. Astra's case for its price does not rest on floorplans; it rests on the professional-work evaluations in O​penAI's launch post, where it is the model with a 1,050,000-token context window and the frontier scores on Agents' Last Exam and Terminal-Bench-Science. If your workload is apartment photographs and connectivity graphs, the price difference is hard to justify from this board alone. If your workload is a long legal or scientific document set, this benchmark says nothing about it either way.

One more caveat on cost: neither Andon nor Artificial Analysis publishes a cost-per-task figure for Blueprint-Bench 2, and this benchmark is unusually sensitive to it. Fifty apartments at roughly twenty images each is a lot of input tokens before the model writes a single room, and a model that reaches the same graph score with more turns of the notepad loop costs more per apartment than its token price implies. Do not read the token prices above as a per-apartment bill; they are not one.

Where this leaves a routing decision

The practical version of all this is that "which model for spatial work" just became a two-model question with a price spread, and the routing layer is where the spread gets resolved. Gemini 4 Carbon is in the OrcaRouter catalogue at list price with 0% markup passed through, which means the $2 and $10 above are what you actually pay per million tokens rather than a vendor rate with a platform margin stacked on top. That matters here specifically because the cost gap between first and tied-second on this board is 4x, and a 4x gap is the kind of thing you want to measure on your own floorplans before you standardise on either.

GPT-6.2 Astra is not on our catalogue as of 20 November; the OpenAI models we serve are GPT-6 Astra, which scores 0.497 on this board, and the GPT-6 Sol line. That is worth saying plainly rather than implying otherwise, because it changes the recommendation: until Astra is routable here, the model on our catalogue that ties the human baseline is Carbon, and GPT-6 Astra sits on the same key if you want the O​penAI output style at $10 and $50. When a model and its cheaper rival land on one endpoint, an A/B on your own apartments costs an API key change rather than a second contract, and failover means a rate limit on one does not stop the batch.

What would change this read

Three things, in order of how much they would move the story. The first is a wider human baseline. Twelve apartments is a small sample for a fifty-apartment task, and Andon publishing an interval on the human figure would settle whether 0.586 is a line or a smear — if the human band turns out to run from 0.55 to 0.62, then Astra's lead is real but Carbon's tie is not, and the whole top of the board reshuffles. The second is a second run of GPT-6.2 Astra on the same board. The model is three days old; Andon's numbers come from one campaign, and benchmarks with a notepad loop and a fifty-apartment horizon have more run-to-run variance than a single-shot eval, which is what the standard-error bars are gesturing at but cannot fully absorb.

The third is the page itself. Andon's "Key findings" section still names GPT 5.5, Gemini 3.1 Pro and Claude Opus 4.7 as the top three models, which was true in May 2026 and has not been updated as the table filled in. Their spatial-reasoning writeups still quote Gemini 3.1 Pro and GPT 5.5 working through individual apartments. The leaderboard is current; the prose around it is a snapshot from six months ago. Read the table, not the findings section, and read the error bars on the chart rather than the caption that says two models tie a line one of them is measurably above.

None of that makes the result wrong. GPT-6.2 Astra is the best model on both independent 3D evaluations published this week, by a margin that clears its own error bar on one of them and by 47 Elo on the other, where the two 95% intervals still overlap. It is the first model to sit meaningfully above a human estimate on a benchmark that has been running since May. The claim is smaller than the tweet and sturdier than a leak, which is the right shape for a result that arrived three days after the model did.

Compared in this article2

Detected from this article · Benchmarks: Artificial Analysis · updated daily