PAN 2 Makes "What If" Watchable — and Says So Where It Fails

Drop a strawberry into a bowl and you already know, without thinking about it, roughly how the liquid jumps. PAN 2 is the world model from the Institute of Foundation Models at MBZUAI that was built to know the same thing and to show it. Give it a starting frame, describe an action in text, and it returns up to twenty seconds of 704×1248 video at 24 frames per second showing what that action does to the scene. The demonstration that went around on 11 November 2026 is the clearest explanation of the idea anyone has produced: swap the strawberry for a rubber duck, or change the liquid, and the splash changes with it, because the model is predicting a physical consequence rather than retrieving a clip. IFM's own announcement, posted the evening before, put the pitch in one sentence — "A language model predicts the next word. PAN 2 predicts the next world: what happens after each action, grounded in time, physics and space."

What makes that more than a slogan is the release wrapped around it. The weights are Apache-2.0 and have been on Hugging Face since 7 October 2026: 86.17 gigabytes across sixty files, with inference code following on GitHub on 5 November and Day-0 serving support merging into vLLM-Omni on 10 November. So the question worth answering is no longer whether PAN 2 exists or who can get it. It is what a downloadable 29-billion-parameter simulator does well enough that you would plan against its output, and where the team's own documentation tells you not to.

The demo is a two-stage pipeline wearing one model's name

The strawberry clip looks like a text-to-video generation. It is not, and the distinction is the whole design. PAN 2 implements the Generative Latent Prediction pattern set out in Critique of World Models (arXiv 2507.05169), which splits prediction into a reasoning half and a rendering half.

The reasoning half is a language model — any instruction-following one — that reads the observation plus the requested actions and writes the next world state as a structured, time-grounded caption: the events in order, each action bound to the actor performing it, plus camera motion. That caption is not decoration. It is the actual interface. A frozen Qwen3.5-9B text encoder maps it into conditioning features, and a 29B multi-modal diffusion transformer trained from scratch on hundreds of millions of videos annotated in the same format fills in the perceptual detail the caption leaves open. A Wan 2.2 VAE decodes the latents into frames.

Which means the demo's real claim is not about pixels at all. It is that a structured description of an action is a sufficient control signal for a simulator to produce a physically coherent consequence — and that an agent can therefore simulate several candidate plans, compare them, and act on the best one. That is what IFM means by simulative reasoning, and it is why the strawberry, the duck and the swapped liquid are three runs of the same question rather than three pieces of footage.

The practical consequence is unfriendly. You cannot type a wish into PAN 2 and get a video. The repository ships a prompt-rewriter skill — a system prompt, a fixed negative prompt, and a validator that checks captions against the text encoder's budget of 1,480 tokens soft and 1,536 hard — and the model card says plainly that the agent case studies relied on hand-refined prompts. Free text gets rewritten before it is used. That is a model with an interface contract, not a chat box.

What the report measured, and the words it used

IFM published more evaluation than most video-model releases do, and labelled the conditions more precisely than most. All of it is the team's own work on the team's own harness, and the model card says so in the section where it matters.

• Single actions — 300 test cases across six capability categories, 201 image-to-video and 99 text-to-video, accuracy reported as a percentage.

• Multi-step plans — 100 egocentric everyday tasks posed both ways, 200 prompts and 456 steps, scored on step completion and full-plan completion, averaged over three seeds.

• Public benchmarks — Physics-IQ Verified on the image-to-video setting, normalised so that the variance between two real takes equals 100; PhyGround on a five-point scale judged by Qwen3.8-27B; RBench as an unweighted mean over five task and four embodiment categories, multiplied by 100.

• Human preference — blind pairwise comparison on 140 prompts, 90 text-to-video and 50 image-to-video, judged on action adherence, physics and world consistency by 13 reviewers across 1,400 comparisons, with Elo fitted by Bradley–Terry and centred at 1500.

Thirteen reviewers is a small panel, and they were judging the authors' model against the authors' baselines. It is still more evidence than most releases in this category publish, and the framing is unusually careful: the closed models in the comparison were generated once through a third-party inference API, and the card flags which rows are single-seed. None of that makes the numbers independent. It makes them legible, which is the next best thing.

Two minutes of GPU per thirteen seconds of video

The number that decides whether you can use PAN 2 is not a benchmark score. It is the one in the vLLM recipe, which states that thirteen seconds of 704×1248 video takes roughly 119 seconds on four B300 GPUs, with classifier-free guidance split across two ranks and Ulysses sequence parallelism across two more. The documented default is eight H200s in BF16, with VAE patch parallelism across four tiles on top, because decoding a twenty-second 704×1248 clip is its own memory problem.

Run the sampler settings against that figure and the cost becomes structural rather than incidental. Fifty steps at a flow shift of 9, with guidance of 2.5 applied on every step, means a hundred forward passes through a 29B transformer for every clip. That is why guidance has to be parallelised rather than run sequentially, and why the honest question about PAN 2 is not "how good is it" but "what does one simulation cost me, and how many simulations does a plan need". A planner that evaluates five candidate actions per step, over a ten-step task, is asking for fifty clips. At the vendor's own published configuration that is a four-GPU node busy for the better part of two hours, before anything has been learned from the output.

The planner is the cheap half, and IFM left it interchangeable

There is one part of this pipeline that does not need a GPU cluster, and the report is explicit that it is swappable. The conceptual reasoner is defined as any instruction-following language model, and IFM ranked four of them by how often a simulated plan completes end to end: Claude Opus 5.5 first, then GPT-6 Astra at medium effort, then Kimi K3, then Qwen3.8-Flash.

All four of those models are in the OrcaRouter catalogue, which makes the reasoner slot the one piece of a PAN 2 stack you can benchmark without renting anything. Point the prompt-rewriter at each of them through one API key, keep the diffusion half pinned to your own hardware, and let automatic failover absorb a provider wobble mid-experiment instead of aborting the run; pricing is each provider's list rate passed through at 0% markup, so a vendor price change shows up at the endpoint the day it happens rather than after a repricing cycle. OrcaRouter does not host PAN 2 and cannot route it — the renderer stays yours — but the half of the pipeline that decides what to simulate is ordinary metered text work, and it is the half where quality actually varies between runs.

Worth saying plainly: the ranking above is IFM's, measured on IFM's tasks with IFM's harness. It is a sensible place to start a shortlist. It is not a substitute for running your own prompts through the four candidates and counting how often your plans survive.

The limitations section is the most useful page in the release

Most model cards put their admissions in a paragraph nobody reads. PAN 2's puts the number up front: about two in three simulated plans contain at least one step that is not carried out. That single sentence reframes everything above it. A simulator that drops a step in two out of three plans is not yet something you hand a robot and walk away from — it is something you use to rank candidate plans whose failure modes you can survive, with a checker in the loop.

The rest of the list is equally specific, and equally worth taking at face value.

• Long-horizon consistency — each clip is simulated from at most one starting frame plus text, so appearance drifts across clips in long interactions; the card names perceptual memory as the missing piece rather than claiming it is solved.

• Prompt sensitivity — wording of the observation and action changes the outcome, and the examples beyond the training distribution needed hand-refined prompts.

• Coarse text actions — text expresses "open a drawer" well and "place each finger here" badly, which bounds dexterous manipulation.

• Shortcuts — a simulation can progress while violating physics, the card's own example being pasta appearing in the colander before it is drained.

• Faithful failure is unmeasured — the quantitative study covers open-loop plans designed to succeed; how well PAN 2 simulates plans that should fail is, in the card's words, not yet measured. That is the gap that matters most for planning, because a simulator that cannot show you a bad outcome is not yet a filter.

Guardrails ship as a separate repository, with numbers attached

The other thing IFM did that most open-weight video releases skip: it published a guardrail alongside the model rather than a paragraph recommending one. pan2-guardrail screens the request with Qwen3Guard-Gen-4B before generation and the generated video with a Q16 frame classifier afterwards, and the card reports what that costs in both directions — harmful text-to-video requests producing unsafe output fell from 33.9% to 1.5%, no unsafe image-to-video outputs were observed, and 11.5% of benign text-to-video requests plus 3.3% of benign image-to-video requests were stopped on the way through.

Those are the team's internal measurements, not an audit, and the false-positive rate is the number to stare at if you intend to run this at volume: a guardrail that rejects one benign request in nine is a design constraint on your product, not a footnote. It is also, to be fair, the kind of disclosure that lets you make that decision at all.

What would make this more than an impressive demo

Three things, in ascending order of difficulty. A failure study: the model card admits that simulating plans which should fail is unmeasured, and until someone publishes that, PAN 2 is a generator of plausible successes rather than a predictor of consequences. An independent evaluation: every capability figure in this article originates with IFM, on IFM's harness, with a thirteen-person panel for the human study, and no third party has reproduced any of it. And a cheaper serving path: the published configuration is a four-GPU node for two minutes per thirteen seconds, which is fine for research and brutal for the loop that simulative reasoning actually implies.

PAN 2 is nonetheless the most complete open world-model release of the year, and the completeness is the point. Weights, code, a serving recipe, a prompt contract with a token budget, a guardrail with false-positive numbers, and a limitations section that leads with the failure rate. You can disagree with its readiness for a robot and still use it this week to ask what happens if you change the liquid — which, for a model whose entire premise is that consequences can be previewed, is a fair first test.