Title card for ERNIE 5.5 Flash reading 'ERNIE 5.5 Flash: A Blackwell-Only FP4 Path Appears Before the Chat Model Does', with chips 'SGLang PR #51236 · opened 18 Nov 2026', 'MXFP4 experts, SM100 only', 'Chat template renders in Python', 'Chat checkpoint: not released'. Footer reads 'Source: sgl-project/sglang PR #51236, open and unmerged.'
Guides & Insights

ERNIE 5.5 Flash: A Blackwell-Only FP4 Path Appears Before the Chat Model Does

Author

jinhao.song

Date Published

Latest models · 20View all models →
Benchmarks: Artificial Analysis · updated daily
Back to all posts

The most concrete thing published about ERNIE 5.5 Flash this week is an error message. Inside a 525-line pull request opened in the SGLang repository on 18 November 2026 is a hard refusal: --moe-a2a-backend megamoe is rejected on anything that is not an SM100 GPU, because the kernels that would otherwise run compute SwiGLU and this model's experts use SiTU. That line, and the rest of PR #51236, is the first public description of how the chat release of ERNIE 5.5 Flash — as opposed to the ERNIE 5.5 Flash Base checkpoint Baidu published on 5 November and the ERNIE 5.5 Flash Preview endpoint it opened on 11 November — is meant to be served. The model itself is not out. Its serving contract now is.

This is a what-we-know-so-far piece, not launch coverage. Everything below comes from one open, unmerged, unreviewed pull request, from the SGLang documentation that was written alongside the earlier Base support, and from Baidu's own published pages. Nothing here has been run against a downloadable chat checkpoint, because no such checkpoint is downloadable. Where a number is Baidu's, it is labelled as Baidu's.

What the pull request actually adds

PR #51236 is titled "[Feat] ERNIE-5.5-Flash: Add Python chat template encoding and FP4 MegaMoE serving" and was opened by chang-wenbin, a contributor, at 12:12 UTC on 18 November. It touches 14 files, adds 525 lines and deletes 24. It has no approvals, its CI runs are all skipped pending a label, and both the accuracy-test and speed-test sections of the description are the unfilled template comments. Read as evidence, that matters: this is code its author believes is needed, not code anyone has verified.

It is the fourth ERNIE 5.5 pull request in a fortnight. #48546 added deployment recipes to the SGLang cookbook and merged on 5 November, the day the Base weights went up. #48547 added the model itself — Ernie5_5_MoeForCausalLM, the attention backend, the KV-cache pool, a DSpark draft module — and merged on 6 November. #49807 extended NVIDIA LiteTopK prefill to "LongCat, ERNIE 5.5, HY-4-family, Kimi K3.1" on 11 November and is still open. The 18 November PR is the first of the four that is not about the Base checkpoint at all.

The chat template is Python, and that changes who can run it

ERNIE 5.5 Flash checkpoints can carry their chat template as Python rather than as a Jinja string, in a tokenizer class the PR names as ErnieChatTokenizer. SGLang's normal path renders prompts by calling apply_chat_template against a Jinja template in the tokenizer config; the new ernie5_5 chat encoding calls the checkpoint's own tokenizer instead, and resolves itself automatically from the architecture string or the ernie5_5_moe model type. The practical consequence is in the PR's own words: the tokenizer has to be loaded with --trust-remote-code, and serving chat with --skip-tokenizer-init is refused outright with an explanatory error.

That is a deployment constraint a reader can act on. A model that renders prompts in Python executes code from the checkpoint repository at request time, so anyone running it air-gapped, inside a hardened image, or behind a policy that forbids remote code has to either vendor the tokenizer or accept the flag. The Base checkpoint does not have this problem: its published tokenizer_config.json carries no chat template at all and declares a plain PreTrainedTokenizerFast, which is what you would expect of a checkpoint that is served through a completions endpoint.

Three smaller details in the same file are worth more than they look. The adapter supports inline system messages — mid-conversation system turns rendered in place, not merged into the first message — which only two other encodings in SGLang's tree do. It uses the renderer's native continuation for continue_final_message, because the generic prefix-append path would drop the prefix inside the opened think block. And it deliberately drops chat_template_kwargs the renderer would reject, since ErnieChatTokenizer raises on an unknown keyword where a Jinja template silently ignores one — the PR names a frontend sending DeepSeek-style thinking as the case that breaks.

The bug fix that follows from this is the most revealing single line in the diff. SGLang's request handler reads a thinking key and defaults it to off; the ernie55 reasoning parser reads enable_thinking and defaults it to on. Injecting the handler's default therefore rendered a thinking-off prompt under a reasoning parser that expected reasoning, and the model's entire answer was parsed into reasoning_content with content left empty. That is exactly the failure mode users of the Preview endpoint have been reporting since 11 November — plain answers arriving as empty content — and the fix implies the chat checkpoint thinks by default.

Reasoning and tool calls get their parsers auto-selected too: ernie55 for reasoning, reading <ernie:think> tags, and ernie55 for tool calls, reading <ernie:tool_call>. The adapter's docstring states that text, tool calling and the thinking toggle are supported and that the template's multimodal branches are "left for a later upgrade" — so as of this PR the chat model is served as what Baidu has described it as everywhere else: a text model.

Why four-bit experts lock the fast path to Blackwell

The second half of the PR is about routed experts stored in MXFP4, run as w4a8 — four-bit weights, eight-bit activations — through DeepGEMM's MegaMoE all-to-all across GPUs. Two implementation choices carry information about the model itself.

• The quantisation is auto-detected from the safetensors header, because the checkpoint carries no routed_experts_quant_method key. That checks out against the published Base config, which has no such field; the loader reads the expert dtype out of the file headers and sets is_fp4_experts from that.

• The MegaMoE path is refused on SM90 and on ROCm, not degraded, because the SM90 FP8 and ROCm MegaMoE kernels hardcode SwiGLU while these experts are SiTU-activated. The PR's comment pins the exact constants DeepGEMM bakes in: SiTU beta 4.0 and linear beta 25.0. The Base checkpoint's own config.json declares activation_situ_beta: 4.0 and activation_situ_linear_beta: 25.0, so the kernel constants and the weights agree — which is the sort of cross-check that makes a leak read as engineering rather than speculation.

The failure mode being prevented is the quiet one. An SM90 kernel that computed SwiGLU on SiTU weights would not crash; it would return fluent, slightly wrong text. Refusing to start is the better outcome, and it is the reason the FP4 story is hardware-gated rather than universal.

What that means commercially is narrower than "ERNIE 5.5 Flash needs Blackwell." It does not. The SGLang cookbook, written for the Base checkpoint, publishes single-node recipes on B300, GB300, B200, GB200 and H100, and the H100 FP8 recipe is eight GPUs because FP8 weights are roughly 280 GB. The FP4 experts path is the one that only exists on SM100 parts, and SGLang has not published a memory table for an FP4 checkpoint because it has not seen one. Treat any "fits on one B200" arithmetic you see — including the obvious estimate that halving expert bytes takes a 280 GB model to something near 150 GB — as an estimate, not a recipe.

What is actually downloadable today

• ERNIE 5.5 Flash Base — open weights since 5 November 2026, in FP8 and BF16, on Hugging Face and ModelScope. 280B total parameters, 14B active, 43 layers (one dense FFN, 42 MoE), 512 routed experts with top-10 plus one shared expert in a 2,048-dimensional latent space. Hybrid attention: 37 layers of heavily compressed attention, 6 of MLA with DSA sparse attention. Licence is the ERNIE 5.5 Flash Base Model License Agreement v1.0 — not Apache 2.0.

• ERNIE 5.5 Flash Preview — API only since 11 November, on Baidu Qianfan, listed with a 1M-token context and rate limits of 300 requests and 3 million tokens per minute. Baidu's release post says the weights "will be open-sourced under Apache 2.0 soon." As of today they have not been. A free variant of the Preview has been callable through a third-party catalogue since 10 November.

• ERNIE 5.5 Flash — the model this pull request is about. Not released. SGLang's cookbook lists it as "ERNIE-5.5-Flash (FP8 / FP4 / BF16, coming soon), with reasoning, tool calling and a bundled DSpark draft," and Qianfan's model list contains exactly one ERNIE 5.5 entry, ernie-5.5-flash-preview. There is no Artificial Analysis page for any ERNIE 5.5 model, and the arena leaderboard's newest ERNIE row is ERNIE 5.1 at 1467 — so every capability number attached to this model remains Baidu's own.

The SGLang cookbook page for ERNIE-5.5-Flash, showing the model-introduction section that lists ERNIE-5.5-Flash-Base in FP8 and BF16 as released, and ERNIE-5.5-Flash in FP8, FP4 and BF16 marked 'coming soon', above the GPU recipe grid running from two B300s to eight H100s.

Which is the honest reading of the whole PR: it is written for a checkpoint nobody can download. The FP4 detection code, the DSpark draft loader that builds main_proj with the FP8 quant config and quantises a BF16 feature_projection at load, and the encode-table warm-up that pays a measured 0.8-second cost at startup instead of on the first request — all of it targets an artefact referenced in a code comment only as "the 1110 checkpoint." Baidu has announced that artefact and has not published it.

A single-column scoreboard titled 'ERNIE 5.5 Flash — what the serving PR reveals', with six rows: Chat checkpoint state 'not released, listed coming soon', Chat template 'Python (ErnieChatTokenizer), needs --trust-remote-code', Expert format 'MXFP4 routed experts, run as w4a8', Fast path 'MegaMoE on SM100 only; refused on SM90 and ROCm', Reasoning format '<ernie:think> tags, thinking on by default', and DSpark draft 'bundled, loader being fixed'. Footer reads 'All rows from SGLang PR #51236 (open, unmerged) and SGLang docs; Baidu figures unaudited.'

What this is worth to someone choosing a model today

Very little as a buying decision, and a fair amount as a scheduling one. The release that would make ERNIE 5.5 Flash a candidate is not this pull request; it is the chat checkpoint plus whatever Baidu prices it at plus one independent evaluation. What the PR does settle is the shape of the deployment you would be signing up for if you plan to self-host: a Python tokenizer that needs remote code, an FP4 fast path that requires SM100 hardware, a DSpark speculative draft bundled into the same checkpoint family, and a reasoning parser that assumes thinking is on. Teams that were assuming "we have H100s, we will run the cheap quantised version" now know the cheap quantised version is the one their hardware cannot take.

If you would rather not bet a production path on either the endpoint or the unmerged serving code, the sane shape is to keep the model behind a routing layer while it proves itself. OrcaRouter's catalogue does not include any ERNIE model — the models list returns 218 entries and no Baidu row — so this is not a claim that you can call it here today. What the platform gives you for a model in this state is the experiment design: one OpenAI-compatible endpoint in front of 200+ other models, automatic failover across providers behind that single endpoint, and 0% markup with provider list price passed through, so a vendor repricing is what you pay the day it happens rather than after a margin is applied. If ERNIE 5.5 Flash later lands on a provider you already route to, trying it becomes a model-string change, not a second integration.

The GitHub pull request page for sgl-project/sglang PR #51236, titled 'ERNIE-5.5-Flash: Add Python chat template encoding and FP4 MegaMoE serving', showing the header line 'chang-wenbin wants to merge 1 commit into sgl-project:main from chang-wenbin:support-ernie55-1118', the tab strip with Checks 133 and Files changed 14, +525 −24, the top of the Motivation section describing ErnieChatTokenizer, MXFP4 routed experts and a bundled DSpark draft, and — further down — the CI States panel listing the Base, Extra and AMD ROCm 10 test runs as Skipped pending a run-ci label.

What would turn this into a release story

Four things, roughly in order of how much they would change the picture. The pull request merging with CI and a reviewed accuracy run would turn "somebody wrote this" into "SGLang can serve it," which is the difference the earlier Base support made on 6 November. An MXFP4 checkpoint appearing on Hugging Face would make the FP4 half of the PR testable by anyone with a B200. The Apache 2.0 weights Baidu promised for the Preview would settle the licence question that the Base release left open. And a first independent score — an Artificial Analysis page, a ranked arena entry, a reproducible agentic benchmark from someone who is not Baidu — would replace the vendor table with evidence.

Until then the interesting question is not whether ERNIE 5.5 Flash is good. It is whether a model can arrive with its serving contract fully specified and its weights still unpublished, and what that does to the people who plan hardware around announcements. This week the answer is yes, and the contract is a chat template in Python, four-bit experts on one generation of GPU, and a refusal to start anywhere else. Baidu's next move is the weights; SGLang's is a review.