ERNIE 5.5 Flash Preview and the "#2 Open Model" Claim Its Own Leaderboard Row Contradicts

Baidu quote-tweeted a leaderboard claim on 12 November 2026 that reads, in full, "Ernie 5.5 Flash Preview by @Baidu_Inc is trending towards becoming the #2 open model on the Code Arena: WebDev, at 1668 pts." Baidu added the parameter counts — 280 billion total, 14 billion active per token — and thanked the community for "putting it to the test." Then it let the rest stand. The problem is that the row producing that 1668 is not labeled open. On the same Code Arena WebDev board, in the same November 10 snapshot, ERNIE 5.5 Flash Preview carries the tag Proprietary, a score of 1668 ±16, the label AutoEval, and votes listed as N/A. The models named as the ones it clears — DeepSeek-V4.1-Flash at 1664 and Kimi K3 at 1658 — are ranked rows with 13,952 and 22,405 human votes respectively, and the D​eepSeek row carries an MIT licence tag. So the sentence is doing three things at once: reporting a real score, promising a licence that does not exist yet, and describing a ranking that the board has not assigned.

None of that makes the number wrong. It makes the number a different kind of object than the one the sentence sells, and the difference is entirely readable in the leaderboard's own columns.

What Arena posted, and the one line Baidu left out

The Arena post went out at 01:08:21 UTC on 12 November 2026, one minute before Baidu's quote. Its full text is worth having in front of you, because the claim and its qualifier are in the same paragraph:

• "Ernie 5.5 Flash Preview by @Baidu_Inc is trending towards becoming the #2 open model on the Code Arena: WebDev, at 1668 pts."

• "The weights will be open-sourced under Apache 2.0 soon."

• "Its score puts it just above: DeepSeek-V4.1-Flash (Max) at 1664, Kimi K3 (Max) at 1658."

Arena hedged the ranking twice — "trending towards becoming" — and attached the open-weights status to a future event with the word "soon." Baidu's quote-tweet reproduced the score and the parameter counts, dropped the Apache 2.0 sentence entirely, and kept the "latest from @arena" framing. A reader who only saw Baidu's post would not know that the open-model half of the claim is a forward-looking statement, because the only place it was stated is the half that got cut.

That is not a misquote. It is a selection, and it moves the burden of proof onto the reader who wants to check it — which is what the rest of this piece does.

The row, read column by column

Code Arena WebDev scores front-end web development work: agentic coding tasks with multiple reasoning steps and tool use, not chat quality and not general coding. Its November 10 snapshot holds 995,980 votes across 158 models. Here is ERNIE 5.5 Flash Preview's row as the board prints it, with no interpretation added:

• Score — 1668, published as 1668 +16/-16, so a 90-ish percent interval of roughly 1652 to 1684.

• Rank — N/A. The row is displayed between rank 16 and rank 17 without a rank number.

• Votes — N/A. There is no human vote count on the row at all.

• Label — AutoEval, a linked tag pointing at Arena's own explainer.

• Licence tag — Proprietary.

• Price — $0.28 in, $1.13 out per million tokens, which matches the Qianfan list price.

• Context — N/A, on a model whose vendor page states a 1M-token window.

Three of those seven columns are N/A, and one of them is the licence column, which is the column the claim depends on. Compare the band it claims to sit above and the picture sharpens rather than softens:

• 1691 — Claude Opus 5 (Max), A​nthropic, Proprietary.

• 1688 — Kimi K3.1 Pro (Max), M​oonshot, tagged "Kimi K3.1 Pro License," a custom licence Arena marks neither open nor proprietary.

• 1682 — Gemini 4 Argon (High), G​oogle, Proprietary.

• 1679 — GPT-6.1 Terra (Max), O​penAI, Proprietary.

• 1676 — Qwen3.8-Max (0902), A​libaba, Proprietary, Preliminary.

• 1673 — GPT-6 Sol (Max), O​penAI, Proprietary.

• 1670 — Qwen3.8-Max, A​libaba, Proprietary, Preliminary.

• 1668 — ERNIE 5.5 Flash Preview, Baidu, Proprietary, AutoEval.

• 1664 — DeepSeek-V4.1-Flash-20261017 (Max), D​eepSeek, MIT, 13,952 votes.

• 1658 — Kimi K3 (Max), M​oonshot, 22,405 votes.

Read that list against the claim. Everything above ERNIE on it is Proprietary except one custom-licence row. The first genuinely open-licensed model you meet is the D​eepSeek row below it. So "#2 open model" is not a description of where ERNIE sits in this table. It is a description of where ERNIE would sit if the weights were published under Apache 2.0 and if AutoEval scores were treated as equivalent to human-voted ones. Both conditions are future or contested, and neither is visible in the quote-tweet.

What AutoEval actually is, and why ±16 matters here

Arena published its AutoEval methodology on 30 July 2026 and last revised it on 12 August. The mechanism is a reward model trained on human preference, used to cast votes automatically: sample live prompt-response pairs, regenerate a response from the target model, have the reward model score both, convert the score difference into a soft probability vote, then combine those synthetic votes with the live human votes on the other models. Arena states the resulting scores are "clearly labeled 'AutoEval,' then updated once enough human votes arrive to validate them," and reports ranking correlation above 0.98 with live evaluation.

That is a defensible way to get a day-one signal, and it is why a model can appear on a board the day it ships. It is also, by Arena's own description, a placeholder. The number is provisional by construction and is expected to move. The label on ERNIE's row is not decoration; it is an instruction to treat 1668 as an estimate pending validation.

Now apply the error bar, which is the part of the row nobody quoted. ERNIE's interval runs from about 1652 to 1684. DeepSeek-V4.1-Flash's 1664 and Kimi K3's 1658 both sit inside it. So does Qwen3.8-Max at 1670, and so does GPT-6 Sol at 1673. The sentence "its score puts it just above DeepSeek-V4.1-Flash at 1664 and Kimi K3 at 1658" is arithmetically true of the point estimates and statistically unsupported as a comparison: on this row's own published uncertainty, ERNIE is not distinguishable from any model between 1652 and 1684. It might be above D​eepSeek. It might be below it. The board does not currently say which.

This is not a technicality aimed at one vendor. A ±16 interval on a board where the entire top-25 spans roughly 200 points means almost every adjacent pair of rows is a tie at this confidence level, and the honest reading of any single adjacent comparison — for any model — is "level." What is unusual here is only that a vendor amplified the adjacent comparison as a result.

Where "#2" comes from, and the licence it depends on

Work backwards from the numbers and there is exactly one way the claim becomes true, which is worth stating because it is checkable and because Arena did not spell it out.

Take the open set to mean "every row Arena does not tag Proprietary." Above 1668, that leaves precisely one row: Kimi K3.1 Pro (Max) at 1688, carrying the tag "Kimi K3.1 Pro License." Everything else above ERNIE is tagged Proprietary. Under that reading, Kimi K3.1 Pro is #1 and ERNIE 5.5 Flash Preview would be #2 — not because it beat the field, but because the field ahead of it is proprietary.

That reconstruction is an inference, not a quotation: Arena's post does not name the model it considers #1, and the board does not expose its open-versus-proprietary classifier beyond the licence tags. If that inference is right, then the claim's load-bearing fact is a custom licence on somebody else's model — Arena treats Kimi K3.1 Pro as open, and the entire "#2 open model" framing inherits that classification decision. If Arena instead counts custom licences as closed, ERNIE's 1668 is the top non-proprietary row and the claim should have read "#1," which would have been a stranger sentence to publish about a preview.

Either way the sentence is doing more work than the data behind it. The number is real. The rank is conditional on a licence nobody has seen.

The distance the framing hides

One more figure belongs in any honest version of this post, and its absence is the most informative thing about it. The top of Code Arena WebDev in the same snapshot is Claude Fable 5.2 at 1836. ERNIE 5.5 Flash Preview at 1668 is 168 points behind that — and 17th of 158 if the AutoEval row were ranked, slotting between Qwen3.8-Max at 1670 and DeepSeek-V4.1-Flash at 1664.

Seventeenth on a 158-model board, one day after release, on 14 billion active parameters out of 280 billion total, is a genuinely good result. It is the kind of result that justifies a launch-week post. It is not the kind of result that needs to be dressed as a podium finish, and dressing it that way invites exactly the scrutiny that costs a vendor more credibility than the honest sentence would have.

What the vendor page does and does not say

Baidu's own release page for ERNIE 5.5 Flash Preview is dated 11 November 2026, states "280B total parameters with 14B active and a 1M-token context," and describes the model as the first post-trained checkpoint on ERNIE 5.5 Flash Base, the base weights published on 5 November. It contains no licence statement, no weights statement, and no benchmark numbers — the page's benchmark references are headlines about earlier ERNIE releases, not scores for this model.

That gap matters for the claim under examination. "The weights will be open-sourced under Apache 2.0 soon" appears in Arena's post. It does not appear on the vendor's release page, and it does not appear in the vendor's own amplification of that post. If the Apache 2.0 commitment is real and current, the release page is the natural place for it and it is not there. If it is real but not yet documented, then the only public source for a licence commitment on this model is a leaderboard announcement — which is a thin place for a licence to live when a vendor is building a ranking on top of it.

How to use this before the numbers settle

The practical question for anyone choosing a model this week is not whether 1668 is a good score. It is whether a provisional WebDev estimate on a preview checkpoint, priced at $0.28 in and $1.13 out with mandatory reasoning billed as output, should displace something already in production. On the evidence available, the answer is no — but the way to test it is cheap, and it does not require believing either the post or this article.

Run the comparison on the routed models instead of on the leaderboard. DeepSeek-V4.1-Flash and Kimi K3 are both available through OrcaRouter behind one OpenAI-compatible key, at provider list price with no markup, so the control group costs the provider's own rate rather than a second contract and a second SDK. Build twenty front-end tasks that represent your actual work — component refactors, a layout bug that only manifests at one breakpoint, a state machine wiring job — run each model twice, and count output tokens as well as correctness. That is the measurement no leaderboard row will give you, because it is the one number that has to be yours: per-task cost, not per-token price.

Treat ERNIE 5.5 Flash Preview itself as the thing being tested rather than the thing being adopted. It is on Qianfan, not in our catalogue, and a preview whose own vendor calls it an intermediate post-training checkpoint is exactly the profile you route behind a primary rather than in front of it. If you do put it on a path, put it on a path with failover, so the day Arena replaces 1668 with a human-voted number — or the day the Apache 2.0 weights appear, or fail to — your users find out from a log line and not from an outage.

What would settle it

Three things, in the order they are likely to arrive, each of which would change the verdict in this article rather than merely embellish it.

• Human votes replacing the AutoEval label on that row. Arena says scores are updated once enough votes arrive. When votes stop reading N/A, the 1668 becomes comparable to the 1664 and 1658 it was set against, and the ±16 question resolves itself. Watch the interval narrow as much as the point estimate move.

• Weights published under a named licence. An actual repository with an Apache 2.0 file settles the open-model question permanently and makes the "#2 open model" framing retroactively fair. A licence announcement without weights settles nothing.

• An independent WebDev measurement. Arena's AutoEval is Arena's own instrument, and every benchmark figure Baidu has published for this model is Baidu's own run on Baidu's own harness. Neither is a third-party result. The first genuinely independent front-end evaluation of ERNIE 5.5 Flash Preview will be worth more than both.

Until then the accurate sentence is the boring one: a preview checkpoint that Baidu shipped on 11 November landed at 1668 on Code Arena WebDev, an AutoEval estimate inside an interval that overlaps the next four models down, on a row the board still tags Proprietary, and its vendor amplified that as a near-term #2 among open models. Every clause of that is checkable today. The version that got tweeted is not wrong so much as unfinished — and it was published as though it were done.