Gemini 4 Pro vs GPT-6 Astra on the Xbox 360 Controller Test — and Why Fable 5.2 Wasn't In It

An X post went up on September 21 carrying three names, a video, and the words "SOTA comparison": Gemini 4 Pro, GPT-6 Astra, Claude Fable 5.2. Two of those names belong to models nobody can call. One belongs to a model that has been on sale for eighteen days. And the artifact the post points at — a game controller drawn as vector graphics in a single shot from a single prompt — was, according to the people who actually ran it, a two-way comparison between the first two. Claude Fable 5.2 was not in it. Neither was Claude Fable 5.1.

That gap between the framing and the evidence is the whole state of the Gemini 4 Pro story right now, and it is worth taking apart carefully rather than dismissing. The controller test is not nothing. It is the single most reproducible-looking artifact anyone has produced about an unreleased frontier model this month, it has a four-month lineage behind it, and it is being read by a lot of people as a verdict. It is not a verdict. It is a draw, from one attempt, that the tester himself could not repeat the next day — and knowing exactly how far it does and does not go is more useful than either the hype or the eye-roll.

The test, and why an SVG became the benchmark

The prompt is older than the models it is now being used to judge. It dates to the Gemini 3.0 era, when it circulated as: "Create an SVG image of an Xbox 360 controller. Output it in a Markdown multi-line code block." That phrasing matters, because it is deliberately hostile to a language model. A vector drawing is all structure and no texture — there is no photographic noise for a model to hide behind, no diffusion sampler to smooth over a mistake. Every curve has to be specified as a coordinate. A controller is a hard target for it: two symmetric grips, a concave dish, shoulder triggers rendered in perspective, sixteen-odd buttons that each need a position, a silhouette that reads as wrong immediately if one proportion drifts. And because the answer arrives as text you can paste into a viewer, the comparison is cheap and portable. Nobody has to trust a screenshot of somebody else's chat window.

In September the community picked the prompt back up. The outputs circulating from the arena sessions — a pelican on a bicycle, a PlayStation 5 controller rendered from five macro camera angles, a 3D pagoda with switchable lighting, a voxel temple scene, an Airbus H145 helicopter page — are described in the reporting as taking roughly eight to twenty minutes each at high thinking effort. The controller is the one that got screenshotted into a comparison.

What the controller comparison actually shows

Here is the result as the tester reported it, and it is a much smaller claim than the timeline has made of it.

• Verdict — a draw. The person who ran it, posting as Lumina, called both outputs "nearly perfect."

• The one asymmetry — Astra's version carried a slightly wrong logo. That is a single-detail edge, and the tester framed it as a single-detail edge, not as a general lead.

• Visual completeness — both renderings show detailed controller bodies, controls and shading. Neither fell apart on the shape.

• What the comparison does not establish — that the unreleased model is ahead. Two outputs judged equally good is the definition of not-a-lead.

• Reproducibility — the output was reportedly produced the previous night, took around twenty minutes, and the tester could not obtain comparable results the next day.

• Identity — the arena entry the output came from was labelled gemini-3.8-flash, a name that already belongs to a model Google shipped on September 2, and was later reportedly relabelled gemini-3.7-flash. Which model was actually behind the alias is unresolved, and two other testers said in the same window that they could not confirm which model they had used either.

Read the list in order and the shape of the evidence becomes clear. The interesting claim is not "the unreleased model won." It is that a model nobody can identify, running under a shipped model's name, produced vector work in one attempt that a careful tester could not distinguish from the output of a frontier model that has been generally available for most of September. That is a real observation. It is also an observation about one afternoon, and the tester said so.

The two-way result and the three-way claim

The post that started this piece says three models. The write-ups describing the same controller session describe two. Claude Fable 5.2's actual status this week is a separate story that happens to share a news cycle, and it is worth being precise about because the conflation is doing work in both directions.

• What is claimed — the X account @chetaslua posted on September 18 that Claude Fable 5.2 "is tested in claude code routing fable 5.1 to it." That is a routing claim about Claude Code, from a single account.

• What the name is — the reporter's own assumption. The tester posting as JAZII, whose September 18 write-up is the most detailed account of the behaviour, states plainly that "Fable 5.2" is a name they assigned, not a name anyone confirmed. There is no model card, no API identifier, no pricing page, no system card, and no second outlet independently reporting a Fable 5.2 route.

• What the corroborated Claude routing story actually involves — this blog has covered it separately: the mid-September silent-routing reports that several independent outlets picked up were about an unreleased Opus-tier checkpoint, not a Fable. The Fable attribution is single-source. Treat it as one account's report, not a finding.

• What the tester did report — at high reasoning effort, the suspected next Fable was preferred over GPT-6 Astra, and was slower and more expensive. That is a preference, from one person, on their own work.

• What is verified about the shipped model — Claude Fable 5.1 has been generally available since September 1, 2026. That model exists, has a price, and can be called today.

So the honest version of the three-way framing is this: two of the three names in that post are unannounced models with community-assigned names, the controller test involved only the one whose identity is at least anchored to an arena entry, and the Fable column is a preference from a single tester who explicitly declined to name the model. That is not a reason to ignore any of it. It is a reason to hold the three names at three different confidence levels instead of one.

The benchmark table doing the actual arguing

The controller SVG is the visible artifact. The thing doing the real persuading is a benchmark sheet that surfaced around September 18 and has been reproduced in Chinese and English tech press ever since. It is worth listing, because the numbers are specific enough to check and none of them have been checked.

• DeepSWE v1.1 — 88.7%, described as roughly two points above GPT-6 Astra.

• GDPval-AA v2 — 2,064 Elo, described as the only entry in the comparison above 2,000.

• Terminal-bench 2.1 — 95.3%, given as the highest score in the table.

• OSWorld-2.0 — 86.8%, placed ahead of both GPT-6 Astra and Claude Fable 5.1.

• Claimed pricing — $2.25 per million input tokens and $11.25 per million output.

• Claimed specs — a 10-million-token input limit, a 256K-token output ceiling, cross-conversation memory.

Every line above is a community claim. Google has confirmed none of it, has not confirmed that the arena entry is Gemini 4 Pro at all, and did not respond to a request for comment. The table carries no checkpoint identifier, no reasoning-effort setting, no agent harness, no tool permissions, no test split, no sample size, and no run log — the things a real evaluation carries and a screenshot of a slide does not. The benchmark names are also drawn from the same family OpenAI used in its own September launch material, which is a tell about where the sheet was assembled rather than where it was measured.

There is a second reason to be careful with any frontier comparison this month: the independent index everyone reaches for was re-versioned to v4.3 on September 7, and re-versioning compresses or stretches the top of the board. A score without its version attached is not comparable to a score from three weeks earlier, ours or anyone's.

What is actually confirmed about Gemini 4 Pro

Strip the claims out and the hard record is short.

• July 21, 2026 — Google confirms it has started its most ambitious pre-training run yet, for Gemini 4. The chief executive repeats it on the Q2 earnings call.

• September 2, 2026 — Gemini 3.8 Flash ships. Google's newest generally available model remains a Flash-tier model.

• September 15–18, 2026 — a leaker posting as "Lentils" claims Gemini 4 Pro checkpoints are appearing internally under the codename "argon"; community trackers log an arena entry under the shipped gemini-3.8-flash label taking eight to ten minutes per answer and producing vector and 3D work well past what that model produces.

• Today — no Gemini 4 model id, no endpoint, no price, no model card. Gemini 3.5 Pro, which was previewed for June and never shipped, was reported by the Wall Street Journal to have been scrapped because its gains were too small.

That is the entire confirmed surface. Everything else in circulation — the table, the price, the context window, the codename, the identity of the arena entry — is inference wearing a decimal point.

What you can actually run while you wait

This is the part the comparison posts leave out, and it is the part that changes a decision.

GPT-6 Astra is not a rumour. It shipped on September 3, 2026, it carries the model id gpt-6-astra, and it is callable today at OpenAI's own list price of $10.00 per million input tokens and $50.00 per million output, with cached input at $1.00. It runs on OrcaRouter at that same provider list price with 0% markup — we pass the vendor's rate straight through, so when a rate moves, the number on our side moves the same day rather than whenever someone gets round to updating a margin. Its OpenAI-reported benchmark lines are DeepSWE v1.1 at 74%, OSWorld 2.0 at 72.6%, Terminal-Bench 4.0 at 57.9%, and ExploitBench at 100%, which is what put it over the Critical cyber capability threshold. Those are vendor-reported and unreproduced, and they are the numbers the leaked Gemini 4 Pro sheet is implicitly arguing against.

Gemini 4 Pro is not on OrcaRouter, and we are not going to pretend otherwise. There is no endpoint, no model id, and no price, because there is no shipped model. Claude Fable 5.1, by contrast, has been generally available since September 1 and is routable at its own list price, as is Claude Opus 5 and the rest of the current frontier tier — one API spans 200+ models at provider list price with zero markup, so a comparison like this one does not require three accounts, three SDKs and three invoices to act on.

If the controller result is what tempts you to restructure a production path around an unannounced model, the sane move is the one the evidence supports: don't. Keep the path you have, and if you want to find out whether the effect generalises to your own work, route a slice of traffic at an unproven path behind a fallback to a model you already trust. That way a bad afternoon costs you a percentage point of traffic rather than an incident, and you get your own answer instead of borrowing someone else's twenty-minute result.

What would make the controller result mean something

• The alias gets a real name. An arena entry is a rumour with a leaderboard row. A model id in a vendor's own documentation is a product. Watch the model list, not the timeline.

• The result reproduces. One tester could not get the same output the next day. Three independent people getting it on demand, with the model identified, is a different class of evidence entirely.

• The leaked sheet gets a methodology. A benchmark score without a checkpoint, an effort setting and a harness is not a measurement; it is a claim about a measurement.

• A price appears. The claimed $2.25/$11.25 would make an unreleased frontier model the cheapest of the three by a wide margin, which is the kind of claim that either shows up in a rate card or does not.

• The Fable attribution gets a second source. Right now it has one, and the person who made it said the name was theirs to begin with.

Questions worth an answer

If the arena entry really is Gemini 4 Pro, why run it under a shipped model's name?

Testing a new checkpoint behind an existing model's label is standard practice for blind benchmarking — it lets a lab collect head-to-head votes without the name influencing the vote. It also means the label tells you nothing about what is behind it, which is exactly the problem here: the alias is evidence that something is being tested, not evidence of what.

Does a draw against GPT-6 Astra mean the unreleased model is disappointing?

No, and this is where the framing does the most damage. A checkpoint that is still in post-training, running behind an alias, matching a shipped frontier model on a hard one-shot task is a much stronger signal than a launch-day chart. It is just a signal about direction rather than about a product you can buy, and the two get read as the same thing in a comparison post.

Should I switch anything today because of this?

Not because of this. Nothing in the controller test changes what is available, what anything costs, or what any model is good at. The decision-relevant facts this week are that GPT-6 Astra and Claude Fable 5.1 are both live and priced, and that Gemini 4 Pro is neither. If you want to hedge toward the possibility, hedge with routing rather than with a rewrite.

The honest read

The controller test is the best evidence anyone has produced about Gemini 4 Pro, and the best evidence anyone has produced is a draw, from one attempt, that did not repeat, on a model nobody can name, reported alongside a benchmark sheet nobody can check. That sounds like a dismissal and it is not. The reason the test spread is that it is the first artifact in this story that looks like a product rather than a slide — a real prompt, a real output, a real person comparing them, and a result that neither side's partisans got to claim outright.

What it does not do is settle anything. If you need to ship this week, the frontier tier that exists — GPT-6 Astra at $10/$50, Claude Fable 5.1 at its own list price — is where your decision actually lives, and both are callable on one key without a second contract. If you want to know what Google has, the only signals that will move this from a what-we-know-so-far piece to a launch piece are a model id, a rate card and a reproduction. Until those arrive, the correct reading of the Xbox 360 controller comparison is the one the person who ran it gave: two very good drawings, and no winner.