
GPT-6.2 Astra Tops the SO-101 Robot Board: 22% Success
- OrcaNEWOrcaCyber: Hy4 Flash Abliterated / Uncensored2026-11-18$0.07 / $0.28 per 1M tokens · 184 tok/s
- tencentNEWTencent: Hy4 Flash2026-11-16$0.02 / $0.07 per 1M tokens · 494 tok/s
- AlibabaNEWQwen: Qwen4 Max2026-11-16$1.50 / $4.50 per 1M tokens · 385 tok/s
- GoogleNEWGoogle: Gemini 4 Carbon2026-11-10$2.00 / $10.00 per 1M tokens · 94 tok/s
- AnthropicNEWAnthropic: Claude Fable 5.22026-11-09$8.00 / $40.00 per 1M tokens · 290 tok/s
- OrcaOrcaCyber: GLM 5.3 Abliterated / Uncensored2026-11-04$3.00 / $5.00 per 1M tokens · 99 tok/s
- OrcaOrcaCyber: Hy4 Abliterated / Uncensored2026-11-04$0.80 / $1.60 per 1M tokens · 132 tok/s
- OrcaOrcaCyber: Kimi K3.1 Pro Abliterated / Uncensored2026-11-04$2.00 / $5.00 per 1M tokens · 86 tok/s
- tencentTencent: Hy42026-11-04$0.83 / $2.50 per 1M tokens · 122 tok/s
- OrcaOrcaCyber: MiniMax M3.1 Flash Abliterated / Uncensored2026-10-29$0.80 / $1.60 per 1M tokens · 104 tok/s
- OrcaOrcaCyber: GLM 5.3 Flash Abliterated / Uncensored2026-10-27$0.80 / $1.60 per 1M tokens · 66 tok/s
- OrcaOrcaCyber: Qwen3.8 Flash Next Abliterated / Uncensored2026-10-26$0.80 / $1.60 per 1M tokens · 89 tok/s
- OrcaOrcaCyber: MiMo V2.6 Flash Abliterated / Uncensored2026-10-26$0.80 / $1.60 per 1M tokens · 41 tok/s
- MoonshotMoonshotAI: Kimi K3.1 Pro2026-10-2447Intelligence
- OrcaOrcaCyber: DeepSeek V4.1 Flash Abliterated / Uncensored2026-10-23$0.80 / $1.60 per 1M tokens · 215 tok/s
- minimaxMiniMax: M3.1 Flash2026-10-1929Intelligence
- OrcaOrca: OrcaCyber Ultra 1.02026-10-17$5.00 / $10.00 per 1M tokens · 109 tok/s
- deepseekDeepSeek: DeepSeek V4.1 Flash 10172026-10-1733Intelligence
- DeepSeekDeepSeek: DeepSeek V4.1 Flash 09102026-10-17$0.15 / $0.60 per 1M tokens · 160 tok/s
- DeepSeekDeepSeek: DeepSeek V4.1 Flash2026-10-1740Intelligence
RoboColosseum's SO-ARM101 leaderboard has a new first place, and the thing sitting on it is not a robot policy. GPT-6.2 Astra went up at 1,984 on the site's DCR scale, with Claude Fable 5.2 second at 1,803, Claude Opus 5.5 third at 1,666, the best fine-tuned vision-language-action checkpoint — Physical Intelligence's pi0.5 — fourth at 1,526, and the previous generation, GPT-6 Astra, fifth at 1,393. OpenAI shipped GPT-6.2 Astra on 17 November 2026; the rating was computed on 20 November at 19:46 UTC and Greg Brockman amplified it on X early the following morning. That is the whole of the announcement. What the quote does not carry is the rest of the row: a 22% full-task success rate, a lead the leaderboard files as inside the error bars, and a sample of 122 blind comparisons taken in scenes the model had never seen.
Read the ranking as a result and you will misread it. Read it as a snapshot of an evaluation still filling up, on the cheapest and least-exercised of the project's three main robot boards, and it says something more useful — about how quickly a general-purpose model closes a gap that specialised manipulation policies spent a year building, and how little of that gap the current sample can actually certify.
What RoboColosseum measures, and what it doesn't

RoboColosseum is a distributed evaluation framework rather than a lab. The paper behind it, arXiv 2611.02497, was posted on 2 November 2026 by a group spanning NUS MAGIC Lab, Hugging Face, EAIC Singapore and BitRobot; the author list includes Dieter Fox, Siddhartha Srinivasa, Thomas Wolf, Harold Soh and David Hsu. The premise is that robot policies cannot be benchmarked from a laptop, so the project runs them on real hardware operated by partners at 124 sites in its initial study, over 194 scenes and 8,237 distinct instructions written on the spot by an evaluator. Every rollout is published on Hugging Face in LeRobot format, failures included.
The metric is where most coverage goes wrong, so it is worth stating plainly. The site keeps two records for every policy: a W/T/L record from blind pairwise comparisons, where a human evaluator watches two runs of the same instruction in the same scene and picks the better one or calls a tie; and a DCR rating, which is fit from full-task success outcomes after calibrating for how hard each task turned out to be. The leaderboard says so itself, in a sentence that has not made it into a single tweet about the result: DCR is "a Difficulty-Calibrated Rasch (DCR) Rating (Elo scale); it should not be interpreted as an Elo rating obtained through sequential Elo updates."
That distinction is not pedantry. An Elo number moves every time a match is played; a Rasch estimate is a joint fit of policy ability and task difficulty, and it changes when either side of that fit changes. Reporting the SO-101 board as "Elo rankings" — which is how the result was described on X — takes a posterior estimate and dresses it as a running scoreboard. The two look alike on a chart, which is exactly why the confusion survives. DCR also requires at least 20 eligible comparisons before a policy is ranked at all, and it deliberately does not use the win-loss-tie record: two policies can have very different W/T/L lines and still be separated by the success-based fit. Hold that in mind when the numbers below appear to disagree with each other.
The board, top to bottom

The SO-ARM101 is the open-source, 3D-printed arm from the LeRobot ecosystem — the one the paper describes as designed to be rebuilt identically at different locations, so that a result collected in Singapore can be reproduced in a lab somewhere else. Ten policies are ranked on it. As of the 20 November refresh:
• GPT-6.2 Astra — 1,984, 122 comparisons, 95 wins / 18 ties / 9 losses, 27 full successes
• Claude Fable 5.2 — 1,803, 112 comparisons, 62 / 23 / 27, 21 successes
• Claude Opus 5.5 — 1,666, 187 comparisons, 98 / 57 / 32, 9 successes
• pi0.5 — 1,526, 194 comparisons, 66 / 57 / 71, 9 successes
• GPT-6 Astra — 1,393, 183 comparisons, 75 / 62 / 46, 5 successes
• Grok 4.7 — 1,366, 181 comparisons, 37 / 79 / 65, 4 successes
• LingBot-MolmoAct2 — 1,259, 214 comparisons, 42 / 92 / 80, 5 successes
• GR00T N1.7 — 1,233, 219 comparisons, 31 / 92 / 96, 3 successes
• MolmoAct2 — 1,122, 216 comparisons, 40 / 83 / 93, 3 successes
• G0.5 — 1,095, 207 comparisons, 18 / 94 / 95, 1 success
Two things fall out of that list immediately. The first is that the top three are all frontier language models driven through a coding harness, not fine-tuned manipulation policies; the best specialist VLA, pi0.5, is fourth. The second is that Claude Opus 5.5 has the most outright wins on the board — 98, three more than Astra — and finishes third, because it also has 57 ties and only nine full successes. Astra's line is the inverse: fewer comparisons, far fewer ties, and three times the completed tasks. Whatever the harness is doing, it is converting partial progress into finished runs, and that is what DCR is built to reward.
Twenty-two percent is the number
Twenty-seven successes from 122 comparisons is 22.1%. That is the highest full-task success rate on the SO-ARM101 board, and it means roughly four out of five attempts do not complete the instruction. The tasks are not exotic — the published rollouts include "Place the cup inside the pan," "pick up the yellow box and put it on the plate," "Rotate the marker by 90 degrees clockwise," "touch the blue pen," and "pick up the fork, and then pick up the knife." Scenes are ordinary tables: NUS COM3, an outside table on a terrace, a spot outside the MAGIC Lab.
That a leader fails four runs in five is the single most important context for the headline, and it is consistent with everything the project has published. The November paper reports that across its six manipulation categories, "no evaluated system exceeds 13% success in any category," and that MolmoAct2, the strongest VLA by expected comparison score, only reached 58% to 77% on the relative measure while its absolute success rates sat between 6% and 13%. The board you are looking at is not a board of robots that can do the task. It is a board of systems that are getting better at not failing completely.
Success rate is also the one figure on the row that cannot be inflated by the metric. A DCR rating depends on the difficulty calibration, which depends on how the fit pools tasks across policies; a success count does not. Twenty-seven is twenty-seven. If you want to compare GPT-6.2 Astra to anything on that list without arguing about Rasch models, compare 27 to 21 to 9 to 5.
The lead is inside the error bars
RoboColosseum draws ±1 posterior standard deviation around each rating, and states in the same line that the leader's "Lead over #2" of +181 points sits "inside the error bars." That is the site's own summary of its own result, printed next to the number rather than buried under it.
Put the ±1 SD bands next to each other and the caution makes sense. Astra's runs from about 1,888 to 2,080; Claude Fable 5.2's from about 1,712 to 1,894. They overlap in a window of a few points around 1,890, and the 181-point gap between the two estimates is about 1.4 combined standard deviations — short of what you would need to call it a lead rather than a tiebreak. Astra's 122 comparisons are enough to rank it first, and not enough to say it is better than the model in second place with any confidence. The gap between first and third is wider and cleaner — 1,984 against 1,666 — but it is also a comparison between a model with 122 comparisons and one with 187, on a board where every other policy has more samples than the leader does.
This is not a flaw unique to the result. It is what a 1,017-comparison board looks like. For comparison, the same project's Franka board has 5,643 comparisons and the Bimanual-YAM board 4,517; SO-ARM101 is the thinnest of the three by a factor of roughly five, with 78 scenes and 70.3 robot-hours behind it. A newcomer can reach first place here faster than on any other board in the set. That is a property of the board, not of the model, and it will keep being true the next time a model ships.
Why a language model is on a robot board
The most interesting structural fact about this leaderboard is that it mixes two kinds of system that are usually kept apart. Five of the ten policies are vision-language-action checkpoints — pi0.5, GR00T N1.7, MolmoAct2, LingBot-MolmoAct2, G0.5 — trained on robot data and, in the Fine-tune Track, fine-tuned on 100 demonstrations per task. The other five are frontier agents: GPT-6.2 Astra, GPT-6 Astra, Claude Fable 5.2, Claude Opus 5.5 and Grok 4.7, each connected through a serving rig and left to write the motion code itself. The paper puts it directly: frontier agents "participate exclusively in the Open Track," while VLA models use embodiment-specific checkpoints.
That design is why the result is legible at all, and why it should not be read as a claim about robotics models. GPT-6.2 Astra has not been trained on SO-101 trajectories. It has not been fine-tuned on the tasks. It is being asked to look at camera feeds, reason about a table, and emit motion commands, which is a test of general competence applied to a physical loop rather than a test of manipulation skill. The fact that this beats a policy trained specifically for the arm is the story; the 22% success rate is the price of doing it that way.
The paper's own cross-embodiment check is the corrective. Policy rankings show "weak agreement across embodiment-specific task suites," with Kendall's W of 0.24 — which is close to no agreement at all. π0.5 ranks first on Franka and Bimanual-YAM and fourth on the G1 humanoid; GR00T N1.7 leads SO-ARM101 after fine-tuning and finishes fourth on the other two. A number earned on one arm does not transfer, and GPT-6.2 Astra has only been run on this one. It is absent from the Franka board entirely, where the previous generation, GPT-6 Astra, currently leads at 1,915.
One day, 122 comparisons
The post that started this says the model was put through "hundreds of head-to-head comparisons in the wild" in "just ONE DAY." The board credits GPT-6.2 Astra with 122. Both numbers can be true — the arena runs comparisons across every policy on it, and the daily throughput is well into the hundreds — but only one of them is the sample behind the 1,984, and it is the smaller one.
The timeline is worth pinning down, because "one day" is doing rhetorical work. OpenAI shipped GPT-6.2 Astra on 17 November. The rating on the SO-ARM101 board carries a computation timestamp of 20 November, 19:46 UTC, and the arena's most recent comparison was logged the same morning at 09:49 UTC. So the 122 comparisons accumulated over roughly three days of a live board, not twenty-four hours of a dedicated run. The site refreshes hourly, which means the number you see is a moving estimate rather than a finished result — and it means a rank change of one or two places is well within what the next refresh can produce.
None of this makes the result wrong. It makes it provisional in a specific, checkable way: the direction is clear, the magnitude is not yet settled, and anyone quoting 1,984 as a fixed quantity is quoting a fit that will move.
What the rank costs

Here is where the noise becomes a purchasing decision rather than a methodological footnote. On the SO-ARM101 board, the top two models list at the same price. GPT-6.2 Astra is $8 per million input tokens and $40 per million output; Claude Fable 5.2 is $8 and $40; GPT-6 Astra, three places down, is $10 and $50. Claude Opus 5.5, in third, is $4 and $20 — half of Astra's input price and half its output price — and pi0.5, the best specialised policy on the board, is open weights and free to serve on your own hardware.
• Rank 1 — GPT-6.2 Astra, 1,984 DCR, $8 in / $40 out per million tokens
• Rank 2 — Claude Fable 5.2, 1,803 DCR, $8 / $40
• Rank 3 — Claude Opus 5.5, 1,666 DCR, $4 / $20
• Rank 4 — pi0.5, 1,526 DCR, open weights
• Rank 5 — GPT-6 Astra, 1,393 DCR, $10 / $50
Read that as a cost curve and the ranking stops being the interesting part. Moving from third to first doubles the per-token bill and buys a lead that the board itself says is inside the error bars. Moving from fifth to first saves 20% and buys 591 DCR points, which is the largest single-generation jump on the board — but it also means the previous generation is now the more expensive option, which is the kind of inversion that only shows up when you look at the price column next to the rank.
For anything in a loop, that arithmetic compounds. A manipulation run is not one request; it is a long sequence of calls, most of them carrying camera frames and a growing transcript, and the per-token price multiplies across every step. A model that finishes 22% of tasks and costs $8 per million input tokens is a different purchase from one that finishes 5% and costs $10, and the difference is not visible in the leaderboard at all.
GPT-6.2 Astra is not on the OrcaRouter catalogue yet, so the honest answer for anyone who wants to call it is the vendor's own API. What the catalogue does carry is the rest of that column: GPT-6 Astra at $10 and $50, GPT-6.1 Sol at $2 and $10, and Claude Opus 5.5 at $4 and $20, all at 0% markup on provider list price, which is why a vendor price cut lands on our side the same day it lands on theirs. Running the same task across three of those and comparing finished runs is the cheapest experiment on this page, and it is the one the leaderboard is asking you to do rather than the one it is reporting.
What would change the verdict
Three things would turn a provisional first place into a settled one, and all three are observable.
The first is sample. SO-ARM101 needs several hundred comparisons per policy before its intervals tighten to the width of the Franka board's. GPT-6.2 Astra's band is 376 points wide at 95% credibility; Franka's leader, the previous-generation GPT-6 Astra, sits in a band about 185 points wide on 194 comparisons. When Astra's sample roughly doubles, the question of whether it is genuinely ahead of Claude Fable 5.2 answers itself.
The second is a second embodiment. A model that leads one arm and has never been run on another is a data point, not a capability claim, and the paper's Kendall's W of 0.24 is the project's own admission that rankings do not travel. If GPT-6.2 Astra appears on the Franka or Bimanual-YAM board at a similar rank, the SO-101 result stops being an artefact of a thin board.
The third is the Fine-tune Track. The paper reports that after fine-tuning on 100 demonstrations per task, GR00T N1.7 leads SO-101 — and GR00T N1.7 currently sits eighth on the open board with 1.37% success and 219 comparisons. That is the most useful line in the whole dataset: the policy that wins when everyone gets the same demonstrations is near the bottom when nobody does, and the frontier agent that wins the open track is not the one anybody would pick to deploy. Two different questions, two different winners, and only one of them is what a robotics team actually needs to answer before it ships.
Until then, the fair summary of 20 November is narrow and worth stating precisely. A general-purpose model, four days old, went to the top of a real-robot leaderboard by finishing 22% of its tasks, on 122 blind comparisons, in front of a field that includes the specialised policies built for that arm — and the site that published the ranking is the first to say the lead is inside the noise.
Compared in this article1
Detected from this article · Benchmarks: Artificial Analysis · updated daily
