Genie 4 vs SKY-001: Two World Models That Will Never Share a Benchmark

Genie 4 and SKY-001 were announced eight days apart in October 2026, and no one will ever publish a fair scoreboard with both on it. Genie 4, from Google DeepMind, generates a playable, audible scene you walk around for about two minutes. SKY-001, from Skyfall AI, takes a prompt or a sketch and emits an ordered construction sequence for a CAD part or a building floor plan. One produces pixels a human watches. The other produces geometry a machine might eventually build from. The only useful comparison is not which is better — it is what each one would have to prove before you put it in front of a paying customer, and which of those proofs has been attempted.

Both companies call their product a world model. That shared phrase is doing an enormous amount of unearned work, and it is why the coverage of these two launches reads as if they were rivals. They are not competing for the same budget, the same buyer or the same evaluation. They are competing for the same word.

The word is doing too much work

A useful way to separate them is to ask what the model is a model of.

Genie 4 models appearance and dynamics. Give it a prompt and it renders a world that responds to typed instructions — change the weather, move the camera, add a person — and now produces ambient sound, music and interactive dialogue alongside the video. A Gemini planner watches the frames and re-plans. The output is a stream of images intended for a human eye or a video pipeline, and its correctness is judged by whether the scene behaves plausibly.

SKY-001 models assembly. Its hierarchy runs from systems to assemblies to subassemblies to parts, or floors to units to rooms, and its second component — Skyfall calls it a construction world model — plans the ordered sequence of visual states that gets you from blank to finished. For a CAD component that is a series of 3D solid states with 2D sketches between them. For a floor plan it is a drawing built stroke by stroke. Correctness is geometric overlap, measured as IoU against a ground-truth shape.

The dimensions, in the form that makes the mismatch obvious:

• Output — Genie 4 emits video and audio; SKY-001 emits a CAD or BIM construction sequence.

• Input — text, keyboard and camera control for Genie 4; text or a 2D image for SKY-001.

• What "wrong" means — a scene that drifts, or a solid that misses the target geometry by an IoU margin.

• Evaluation unit — a human judgement or a physics score for Genie 4; mean IoU across 1,000 cases for SKY-001.

• Published price — neither.

• Access — waitlist on both sides.

Test one: can anyone check the output?

SKY-001 passes this test in the narrowest possible sense, and that is still more than Genie 4 manages.

Skyfall published a vendor benchmark on two tracks of 1,000 cases each, and the numbers are specific enough to argue with. On CAD components, SKY-001 scored 79.7% mean IoU against 79.1% for GPT-6 Astra — a lead of six-tenths of a point, which is seed-level noise — while running in 3.9 seconds instead of 199 and costing $0.005 per case instead of $0.75. On floor plans it scored 43.6% against GPT-6 Astra's 40.6%, at 1.22 seconds versus 144.6.

Every one of those figures is Skyfall's own, produced on Skyfall's internal test set, with no public split and no released harness. The IoU metric measures geometric overlap and nothing else: it does not establish that a part can be manufactured, that tolerances close, that a floor plan meets egress or accessibility code, or that the structure stands. The timings exclude upload, preprocessing, queueing, format conversion, validation and human review, which in a real CAD workflow are most of the elapsed time. And the multipliers in Skyfall's own launch material do not agree with each other — 51×, 49× and 100× for speed, 150× and 143× for cost, across pages of the same site.

Genie 4 does not clear even that bar. Google published no benchmark, no resolution, no frame rate, no latency and no parameter count. The company's own limitations section names the problems — visual detail and object permanence drift, complex physical interactions, audio accuracy, legible in-world text, spatial accuracy — without quantifying any of them. There is no third-party submission and no tester outside the pool has published a measurement. On the single question of whether the thing works, SKY-001 offers a claim you can pull apart and Genie 4 offers a demo reel.

Test two: can the output enter a tool?

This is where SKY-001's architecture earns its keep, and where Genie 4's ceiling sits.

A video of a world is not a world you can edit. Nothing in a Genie 4 session exports as geometry, so a game studio using it for prototyping gets a reference to rebuild by hand, and a robotics team gets observations to train against but not a scene graph to modify. That is a real use — training data and pre-visualisation both consume pixels — but it means the model sits upstream of the toolchain, feeding it rather than entering it.

SKY-001's stated output is the toolchain. The construction sequence is designed to be ingested, and Skyfall demonstrated it working inside Autodesk Fusion. That demonstration is not a shipped connector, and one demo is not an integration story. Independent reviewers have also flagged that the saved sketches and the generated 3D geometry may not align, which would leave output that looks like CAD without being machine-ready CAD. But the intent is different in kind: one model is trying to shorten the distance to a manufacturable artefact, the other is trying to produce a convincing observation.

For a design firm, that distinction decides the purchase. A renderer that saves a concept artist two days is worth buying at almost any price. A sequence generator that saves a drafter two days is worth buying only if the sequence survives contact with the drafter's actual tools — and nobody outside Skyfall has tested that.

Test three: can you buy it before 2027?

No, on both counts, and the timelines are not close.

SKY-001 is a research preview available by waitlist. There is no API, no price list, no weights, no licence, no self-hosting path and no fine-tuning. Skyfall targets commercial deployment in early 2027. The company is small — around 25 people — and its founder, Sam Pasupalak, previously built Maluuba, which Microsoft acquired in 2017. That is a credible pedigree and it is not a shipping product.

Genie 4 is a research preview for trusted testers, with Google AI Ultra subscribers promised access through Project Genie at roughly $100 a month "in the coming weeks", and early-access partners reaching it through the Gemini API and AI Studio. No general-availability date, no developer price, no commercial terms. For a studio, the missing licensing terms are a bigger blocker than the two-minute ceiling, because you cannot put generated footage into a production pipeline whose commercial use is unstated.

So the honest procurement answer for both is the same sentence: there is nothing to buy. The difference is what you can do while you wait. Odyssey-3, the other serious interactive world model of the moment, at least lets you drive a browser demo today. Neither of these two does.

The number in Skyfall's own table that undercuts the category

Buried in the BIM results is the most interesting figure either company published, and it is not the one in the headline.

On Skyfall's 1,000 floor plans, SKY-001 scored 43.6% IoU. Gemini 3.8 Flash — a general-purpose model with no engineering training and no construction world model — scored 42.3%. That is a gap of 1.3 points, on the specialist's own test set, against a model anyone can call today. The speed and cost difference is real and enormous: 1.22 seconds against 346.2, $0.0024 against $0.218 per plan. But if the accuracy lead over an off-the-shelf frontier model is 1.3 IoU points on a metric that ignores manufacturability, then what SKY-001 is selling is throughput, not judgement. That is a legitimate product — a 280× latency win is a different workflow — but it is a much narrower claim than "beats frontier models".

Which reframes the whole matchup. Genie 4 is not competing with SKY-001. Both are competing with the general models you can already route. A design pipeline that calls a frontier model per case, with a validation step and a human reviewer, is the baseline these specialists have to beat, and on the one track where Skyfall published both numbers, the baseline came within 1.3 points.

That baseline is the part you can actually assemble today. Gemini 3.8 Flash is on our catalogue as google/gemini-3.8-flash, and so are GPT-6 Astra and the rest of the frontier field, all behind one API with provider list pricing passed through at 0% markup. OrcaRouter does not host Genie 4 or SKY-001 — neither is an addressable endpoint anywhere — so the practical architecture is a routable model doing the work now, with failover across providers so a single vendor outage does not stop a batch, and the specialist slotted in later if its claims survive contact with your own drawings.

Where the two actually overlap

There is one genuine point of contact: both models plan.

Genie 4's planner is a separate Gemini model in the loop. It observes frames, breaks an objective into steps, checks progress and revises. SKY-001's planner is internal — the construction world model decides the order of states and the hierarchical composer decides what decomposes into what. In both cases the interesting engineering is not the renderer or the refiner, it is the sequencing logic that decides what happens next.

That similarity is also why the comparison flatters both. A model that plans a sequence of states is doing something more useful than a model that emits one artefact, and both companies know it. But the sequences are not commensurable. One is a sequence of appearances, checked against whether a human finds it plausible. The other is a sequence of solid states, checked against whether it overlaps the target geometry. Agreement between them is not possible because there is no shared ground truth to agree on.

Who should be watching which

If you make films, games or interactive experiences, Genie 4 is the one on your roadmap and SKY-001 is irrelevant to you. Watch for a resolution, a frame rate and commercial terms; the audio feature is already differentiated and does not need to be re-proven.

If you draft parts or buildings, SKY-001 is the one to track and Genie 4 is a curiosity. Watch for a released test set, a shipped Fusion connector, and a price. Until the test set exists, the 79.7% figure is a claim about Skyfall's own data, and the 1.3-point BIM margin over Gemini 3.8 Flash suggests the accuracy advantage may be smaller than the marketing implies.

If you are building either product's downstream layer — the validator, the orchestrator, the review tool — you have the better position. Both models are gated, both are unreproducible, and both will need exactly the plumbing nobody has built: something that takes a generated sequence, checks it against a specification, and decides whether to accept it. That layer can be built now, against models you can call today, and it will still be the useful part when the specialists ship.

The trap in both launches is the same trap. A world model is impressive in proportion to how much it looks like the real thing, and useless in proportion to how hard it is to tell whether it does. Genie 4 looks extraordinary and cannot be measured. SKY-001 can be measured and the measurement is its own. Neither is a product yet, and the eight days between their announcements are the only thing that makes them look like rivals.