NVIDIA Nemotron 3.5 Super VL Base BF16: the 249 GB checkpoint NVIDIA listed but never linked

nvidia/N​VIDIA-N​emotron-3.5-Super-VL-120B-A12B-Base-BF16 is the artifact in the N​VIDIA N​emotron 3.5 Super VL release that N​VIDIA never linked. The company's announcement post of October 15 says, in a single clause, that "the release includes base, post-trained, and NVFP4-quantized checkpoints with training recipes, data, and reinforcement-learning environments." Every repository link on that page points at nvidia/N​VIDIA-N​emotron-3.5-Super-VL-120B-A12B-NVFP4. The string "Base-BF16" does not appear in the post at all.

The repository is real and public. Hugging Face records its initial commit at 2026-10-12T15:31:49Z, its weight shards uploaded at 2026-10-13T04:55Z, and its model card's own release-date field reading "Hugging Face - 10/15/2026". It is not gated, it needs no token to download, and it weighs exactly what the post-trained nvidia/N​VIDIA-N​emotron-3.5-Super-VL-120B-A12B-BF16 build weighs: 248.9 GB across 63 BF16 shards against the post-trained build's 248.9 GB across 64, the two index files 16 bytes apart. Three days after the weights landed it had been downloaded 4 times, against 142 for the post-trained full-precision repo and 81 for the NVFP4 sibling. Those are Hugging Face's own counters, read on October 16.

So this is not a leak or a rumour. The weights are the wrong thing to be wrong about — they are simply the least publicised artifact in a release that is otherwise loud, and the model card behind them contradicts itself in a way that matters if you are the kind of person who would download a 249 GB pre-post-training checkpoint. That contradiction is below.

What landed, and the sentence that mentions it

Three main repositories and one drafter went up in the same window, all first-party N​VIDIA, all under the OpenMDW-1.1 licence, none gated:

• Base BF16 — nvidia/N​VIDIA-N​emotron-3.5-Super-VL-120B-A12B-Base-BF16, repository created 2026-10-12T15:31:49Z, weights 2026-10-13T04:55Z, README last touched 2026-10-15T15:22:08Z.

• Post-trained BF16 — nvidia/N​VIDIA-N​emotron-3.5-Super-VL-120B-A12B-BF16, created 2026-10-12T15:32:06Z, seventeen seconds after the base repo.

• Post-trained NVFP4 — nvidia/N​VIDIA-N​emotron-3.5-Super-VL-120B-A12B-NVFP4, created 2026-10-12T15:32:30Z, and 75.1 GB in total rather than 248.9 GB.

• An NVFP4 drafter — nvidia/N​VIDIA-N​emotron-3.5-Super-VL-120B-A12B-NVFP4-LiLiCorr, created a day earlier still, at 2026-10-11T20:20:59Z.

The mapping between those repositories and N​VIDIA's prose is where it gets uneven, and checking it is worth doing rather than assuming. The announcement's deployment anchors all resolve to the NVFP4 build. The base card's own opening note points outward to the other two — "Looking for a post-trained model? See the BF16 release for customization or the NVFP4 release for optimized deployment" — so the base repository will tell you where to go instead, in its first line. The reverse does not hold. The post-trained BF16 card contains no link to the base repository anywhere, and neither does the NVFP4 card. The base checkpoint is the only artifact in the release that N​VIDIA's own documentation never routes you to; you find it by reading the announcement's nine-word clause and then browsing the org's file listing, which is how it was found here. Nothing in N​VIDIA's copy explains why a base checkpoint is published at the same 249 GB as the model that is actually post-trained, or what you are meant to do with it that the post-trained one cannot do.

That absence is the story. An open-weights release where the vendor links two of three checkpoints and describes the third in nine words is not hiding anything — the weights are right there — but it does mean the documentation you need is not in the announcement. It is in the repository.

What "base" means in this release

The two model cards answer this, and the difference between them is a clean read on what post-training bought here.

The base card documents three training stages. Stage 1 builds the language backbone on top of N​emotron 3 Super Base and trains a shared MTP head for 100B tokens against a frozen backbone, then distils for 1T tokens from N​emotron 3 Ultra Base. Stage 2 is 100B tokens of Indic continued pre-training, a 35B-token extension to 1M context, 67B tokens of software-engineering mid-training at 256K context, and a 9B-token long-context recovery stage. Stage 3 bolts on the C-RADIO vision encoder and the vision-language connector.

The post-trained card documents six, and the three that the base card lacks are the three that make a model usable: Stage 4 supervised fine-tuning on vision, reasoning, code, tools, chat, safety and long-context data, using packed sequences up to 524,288 tokens; Stage 5, reinforcement learning with verifiable rewards plus RLHF plus multi-teacher on-policy distillation; and Stage 6, boosting the MTP head against the frozen post-trained backbone so speculative decoding actually pays off.

The base card says so itself, in the Inference section: "The Base model is intended as a starting point for further training. For chat, reasoning, and tool-calling applications, use the post-trained BF16 or NVFP4 release and its deployment instructions." Its stated use case is "developers and researchers building custom vision-language models through continued pre-training, supervised fine-tuning, reinforcement learning, and distillation."

That is an unusually direct disclaimer, and it is the correct reading of the artifact. What ships as "base" here is not a weaker chat model. It is the substrate that Stage 4 through Stage 6 were run against, published alongside them so that somebody else can run a different Stage 4 through 6 — on their own data, under their own licence terms, without re-running a 20-trillion-token pre-training.

The 1M-token claim the config does not carry

This is the discrepancy worth the 249 GB of your attention.

The base model card states, in its Description, that the model "supports context lengths up to 1M tokens", and repeats it in the Model Design section: "The model supports up to 1M tokens of context." The training methodology backs the intent — Stage 2 explicitly includes "a 35B-token extension to 1M context, followed by a 9B-token long-context recovery stage."

The shipped config.json says something different. Its max_position_embeddings is 262,144 — 256K, not 1M. There is no rope_scaling block and no original_max_position_embeddings field, so the position scheme is not declaring an extension either. The post-trained repository's config, by contrast, declares max_position_embeddings of 1,048,576. Two sibling checkpoints, same architecture string, same layer layout, and a factor-of-four difference in the position limit that the base card's prose does not mention.

Read the three facts together and the most defensible conclusion is not that the base model is short-context. It is that the base repository ships a conservative declared limit and expects you to change it. The 1M extension was trained. The published config does not expose it. If your continued-pretraining plan assumes a 1M-token sequence length straight out of the box, that assumption comes from the card, not from the file the runtime will actually read — and the fix is a config edit you have to know to make.

Worth flagging for the same reason: the base card's Model Summary table simply does not exist. The post-trained card carries one — total parameters, architecture, precision, context length, deployment target, speculative decoding, reasoning mode, recommended sampling, best-for, licence. The base card goes straight from Description to Licence, then to a benchmark table. There is no recommended sampling row anywhere, and there is no deployment row, so nothing on the base card tells you what hardware it wants.

The architecture, read out of the shipped config

Because the vendor's description of the base architecture is a paragraph and the real shape is in the file, here is what the file says. All of it is first-hand from config.json in the base repository, not from any summary.

• Depth and composition — 88 blocks in the language backbone. 40 are Mamba-2 layers, 40 are mixture-of-experts layers, and 8 are full attention layers, at indices 7, 16, 25, 36, 47, 58, 69 and 78. One attention block roughly every nine layers is the entire long-context mechanism.

• MoE — 512 routed experts per MoE layer, 22 selected per token, one shared expert, a latent size of 1024, and a shared-expert intermediate size of 5376 against 2688 for the routed experts. That is the arithmetic behind the "120B total, 12B active" headline.

• Attention — 32 query heads against 2 key-value heads at head dimension 128, so grouped-query attention with a 16:1 ratio. Hidden size 4096.

• MTP — one next-token-prediction layer, itself composed of an attention block and a MoE block, in the style of D​eepSeek's multi-token prediction head. One prediction step, not several.

• Vision — a C-RADIO ViT-H tower: 32 layers, hidden size 1280, 16 heads, patch size 16, 7 register tokens, 3 class tokens, a maximum image size of 2048 and a patch budget between 1024 and 13312. The connector projects to 20,480, then a downsample ratio of 0.5 and a forced image size of 512 claw the token count back.

• Video — a temporal patch size of 2 and a declared pruning rate of 0.7.

• Two oddities — the config declares an <audio> context token while its sound_config is null, so the plumbing for audio exists in this architecture and this checkpoint carries none of it. And tie_word_embeddings is false, which is standard for a model this size and worth knowing if you plan to resize a vocabulary.

For scale: the repository's own safetensors metadata totals 124,453,395,974 parameters, counted one byte of tensor at a time, against N​VIDIA's 120B-total / 12B-active headline. The two numbers are reported on different bases — the vendor figure is the language model's routed-plus-shared arithmetic, the file figure is everything stored in the repository, vision tower and MTP head included. Do not treat the gap as a discrepancy; do budget for the file.

What the benchmark table actually measures

The base card publishes 24 benchmark rows and, to N​VIDIA's credit, a column that says what each one is:

• MMLU 84.14 exact match · MMLU-Pro 76.25 · AGIEval-En 75.65 CoT · SuperGPQA 47.65 CoT · GPQA Diamond 66.16 at 5-shot CoT exact match

• GSM8K 91.74 at 8-shot CoT · Minerva MATH 87.66 · MATH-500 81.56 sampled pass@1 n=32 · HumanEval 81.31 EvalPlus sanitized · MBPP-Sanitized 79.06 · LiveCodeBench CodeGen 47.61 · BigCodeBench 48.42 complete and 49.39 instruct

• ARC-Challenge 95.99 · HellaSwag 88.96 · TruthfulQA MC2 71.18 · WinoGrande 79.08 · PIQA 85.09 · CommonsenseQA 85.26 · OpenBookQA 48.80 · SocialIQA 49.03 · RACE 91.39

• Global MMLU 86.56 averaged across 8 languages · MGSM 87.53 at 8-shot native CoT across 6 languages

Two things about that table are easy to miss and both matter.

First, the card's own framing sentence says the results "evaluate the N​emotron 3.5 Super text-only base backbone, before vision-language training." Whatever else that means, it means these numbers are not a multimodal scorecard — there is no OCR benchmark, no document-understanding row, no video row anywhere on the base card, even though video and document inputs are described two sections earlier. If your reason for wanting the base checkpoint is its vision tower, the card gives you nothing empirical to plan against.

Second, the table has no baseline columns. The post-trained card's table runs seven columns wide — N​emotron 3.5 Super GA against Qwen3.5-122B-A10B, Qwen3.8-Flash-Next, DeepSeek-V4-Flash0731, N​emotron 3 Ultra, Inkling-Small and GLM-5.3-Flash. The base card has exactly one column. And the mix of what was run is telling: the post-trained card drops MMLU entirely in favour of MMLU-ProX, while the base card runs both MMLU and MMLU-Pro and neither MMLU-ProX nor the agentic and harness-level suites the post-trained card leans on. You cannot answer "is this base better than that base" from the card. You would have to run both.

Every figure above is N​VIDIA's, produced through N​VIDIA's own NeMo Gym harness — the post-trained card says so explicitly, and the base card publishes no harness note at all. Nothing on either table has been independently reproduced.

Base versus post-trained, in the one benchmark they share

The two cards do not share a benchmark table, but they share exactly one row you can put side by side, and it is the one that matters most.

• GPQA Diamond — 66.16 on the base card, at 5-shot chain-of-thought exact match. 89.52 on the post-trained card. A 23.36-point gap, from N​VIDIA's own two documents, with the base figure explicitly labelled as the text-only backbone.

That is the honest headline for what Stages 4 through 6 are worth on graduate-level science reasoning: post-training is not a polish pass on this model, it is roughly half the capability again. It also tells you the direction of the trade. If your plan is to post-train the base checkpoint yourself, you are starting 23 points behind the model N​VIDIA is selling and betting that your data and your reward signal close that gap in your domain — which is exactly the bet N​VIDIA's own customer numbers describe: Scale AI reporting a 40x inference-cost reduction for vendor credit processing, Ramp expecting up to 50% higher throughput and 10x lower latency for receipt-to-JSON extraction. Those are vendor- and customer-stated outcomes with no independent measurement, and they are the only published evidence that closing the gap is achievable.

Note the trap in reading the two cards together. The base card's "MMLU-Pro" at 76.25 and the post-trained card's "MMLU-ProX" at 76.91 look like a 0.66-point gap and are not comparable — different benchmarks, one of them with a stated TigerLab extraction method. The base card's "MMLU" at 84.14 has no counterpart on the post-trained card at all. If you want a same-benchmark delta, GPQA Diamond is the one you get, and LiveCodeBench is the one you do not: the base card reports CodeGen at 47.61 while the post-trained card reports LiveCodeBench v6 at 88.22, which is a different version of the same benchmark family and not a delta.

What 249 GB buys, and what it does not

The download itself is one command, and it is the only command in the entire base model card:

• hf download nvidia/N​VIDIA-N​emotron-3.5-Super-VL-120B-A12B-Base-BF16 --local-dir ./models/super-vl-base-bf16

That is the whole Quick Start. The base card contains one code block; the post-trained card contains twenty-two. There is no vllm serve line, no SGLang launch, no tensor-parallel guidance, no --max-num-seqs, no reasoning-parser flag, no MTP configuration for speculative decoding, and no API client example. The post-trained card carries all of it, including a 4×H100 recipe with --enable-mamba-shared-prefix-checkpoint and a 2×GB200 recipe at TP2.

• Runtime engines — the base card names PyTorch, Megatron-LM and NeMo, and its Inference section names PyTorch alone. The post-trained card names vLLM and SGLang. Those are training-framework names on one side and serving-runtime names on the other, and the split is deliberate rather than accidental.

• Test hardware — Hopper H100 and H200, Grace Blackwell GB200 and GB300, Blackwell B200 and B300. The post-trained card narrows to 4×H100 or 2×GB200 for deployment.

• Disk — 248.9 GB over 63 shards, the same as the post-trained full-precision build. The NVFP4 sibling is 75.1 GB. If you are downloading the base to inspect or adapt it rather than to run it, you are paying 3.3 times the NVFP4 download for the privilege of getting no serving recipe.

What you do get that is genuinely usable: the repository ships a complete chat_template.jinja of 12,513 bytes, and it contains the enable_thinking switch, including the medium-effort variant. It also ships a generation_config.json with temperature 1.0, top_p 0.95 and do_sample true. Diffing that template against the post-trained one gives a 414-byte difference, all of it whitespace handling around the image and video placeholders — the two templates are otherwise the same file.

So the plumbing to talk to the base checkpoint as a chat model is present and nearly identical to the shipped chat model. What is absent is any statement that you should. There is no recommended-sampling row on the base card, no reasoning-mode row, and no benchmark for it in chat configuration. You can serve it as a chat model in about ten minutes. Whether the output deserves to leave a test harness is a different question, and the card's answer is that it does not.

The corpus disclosure nobody is talking about

Here is the part of the base release that is arguably more useful than the weights, and it has had almost no attention because it lives in tables on a repo that was never linked.

The base card publishes the training corpus in detail the post-trained card does not carry: a foundation corpus of more than 20 trillion tokens collected between 2013 and December 2025, itemised by source — N​emotron-CC-v2 and v2.1 at 9.1T, N​emotron-CC-Code-v1 at 427.9B, the N​emotron-Pretraining-Code sets at 1.7T combined, N​emotron-CC-Math-v1 at 133.3B, the specialized and SFT synthetic sets at 660B, and a legal set at 4.3B. English, 19 other spoken languages, 43 programming languages.

Then two tables that most open-weights releases simply do not publish. One lists private, non-publicly accessible datasets owned by third parties, and names the models that produced their labels — Qwen3.5-122B-A10B across a wide set of OCR and visual-question-answering relabelling jobs, D​eepSeek-OCR on a filtered Chinese OCR subset, Qwen3.5 on a verified-correct subset, Gemini 3 and GPT-5 on RL data for search. The other lists N​VIDIA's own private datasets, and again names the generators: Kimi K3 for math chain-of-thought SFT data, DeepSeek-V4-Pro for competitive-coding OCR, gpt-oss-120b for structured-output RL, GLM-5.2 for refinement, MiniMax-M2 for search-graph walks, and a long tail of others.

Then two tables that most open-weights releases simply do not publish. One lists private, non-publicly accessible datasets owned by third parties, and names the models that produced their labels — Qwen3.5-122B-A10B across a wide set of OCR and visual-question-answering relabelling jobs, D​eepSeek-OCR on a filtered Chinese OCR subset, Qwen3.5 on a verified-correct subset, Gemini 3 and GPT-5 on RL data for search. The other lists N​VIDIA's own private datasets, and again names the generators: Kimi K3 for math chain-of-thought SFT data, DeepSeek-V4-Pro for competitive-coding OCR, gpt-oss-120b for structured-output RL, GLM-5.2 for refinement, MiniMax-M2 for search-graph walks, and a long tail of others. The paired tables are a reminder of how the industry actually works — a frontier model's training signal routinely includes outputs from competitors' models, and N​VIDIA is one of very few labs that will say which and for what.

The repository also ships a Nemotron_3.5_Super_VL_EU_Training_Content_Summary.pdf, added October 15, which is the EU-facing training-content disclosure. The mid-training mix is disclosed to the same granularity, and one line of it is directly relevant if you intend to fine-tune for software work: the 67B-token software-engineering stage is 20% whole repositories, 41% issue-and-pull-request pairs, 3% commit diffs, 1% build and CI logs, 10% agent traces, 20% pre-training replay and 5% long-context data, and its sources are decontaminated against SWE-bench and Terminal-Bench repositories. That decontamination statement is the one line in the card that tells you what a score on either benchmark is worth.

The licence, and who the base checkpoint is for

OpenMDW-1.1, the same licence as the post-trained builds, with the model card stating plainly: "This model is ready for commercial or non-commercial use." The repository is not gated and no Hugging Face token is required for the download, though N​VIDIA's own container recipes elsewhere in the family assume one.

Put the licence next to the use case and the intended audience is unambiguous. Nobody downloads a 249 GB pre-post-training checkpoint to save money on inference — the NVFP4 build is 3.3 times smaller, faster, and 23 points better on GPQA Diamond. The base checkpoint is for a team that intends to own the post-training stage: a different domain, a different reward signal, or a different data policy than the one N​VIDIA ran. Under OpenMDW-1.1 with commercial use permitted and weights ungated, that is a legitimate thing to build a product on, which is precisely why the missing documentation costs something.

And it is worth separating the two kinds of reader here. If your question is "what should I call this week for document parsing or video understanding," the base checkpoint is not your answer and never was; the post-trained model is, and the open question there is benchmark verification rather than availability. If your question is "can I get a 120B-class hybrid Mamba-MoE vision-language backbone with a permissive licence as the starting point for my own training run," then this repository is the answer and it has been sitting unlinked since October 12.

If you were going to evaluate it

The expensive mistake with a checkpoint like this is treating the download as the test. It is not: 249 GB, 63 shards and a config edit before the first forward pass tells you nothing about whether the vision tower transfers to your document class.

The cheap test is to establish the capability question on a served model first and only then spend the disk. A routing gateway is useful for that and not for anything else in this decision — one key and one endpoint across a couple of hundred models at provider list price with 0% markup passed through means you can put the post-trained N​emotron 3.5 Super VL build against whatever vision model you are already using, on your own images, in an afternoon and for the cost of the tokens. Automatic failover across providers matters here for a narrow reason: an evaluation run that dies at request 4,000 of 5,000 costs you the whole run, and a gateway that retries elsewhere on a provider error is the difference between one afternoon and two.

To be direct about the one thing that would otherwise be ambiguous: N​VIDIA N​emotron 3.5 Super VL is not on our catalogue. We do not serve this model today, and the base checkpoint specifically is not something a hosted endpoint is likely to carry at all, because base checkpoints are not inference products. What a gateway gives you is the cheap half of the experiment — the comparison that tells you whether paying the 249 GB is worth it.

What is still unestablished

Four things, stated as gaps rather than risks.

• Which context limit is real. The card says 1M, the shipped config declares 262,144, and the training description says a 1M extension was run. There is no published statement resolving the three. Anyone planning a long-context continued-pretraining run should treat the config value as the default and verify against their runtime rather than the card.

• What the vision tower can do. The base card publishes no multimodal evaluation of any kind. No OCR, no document, no chart, no video row — for the one checkpoint in the family whose entire purpose is to be fine-tuned on those modalities. That is the largest hole in the published evidence.

• How the base compares to anything. Single-column benchmark table, no baselines, no harness note. The 66.16 GPQA Diamond figure is reproducible in principle through NeMo Gym and has not been reproduced by anyone outside N​VIDIA.

• Whether the base ships the 1M deployment recipes anywhere. The post-trained card carries 4×H100 and 2×GB200 launch commands for the full window. The base card carries one hf download. It is not clear whether that is a documentation gap or a statement that serving the base at 1M is not something N​VIDIA intends to support.

None of that makes the release less interesting. It makes the base checkpoint the family's least-documented artifact and, for a narrow and specific kind of team, its most useful one — the difference being whether your next step is an API call or a training run.

For a model whose failure modes are still undocumented, the useful counterpart is the record of what has already gone wrong elsewhere: the documented AI incidents and agent failures around models in this class are collected in the OrcaRouter AI Incident Archive, where each entry carries its own sources, severity, confidence grade and disputed flag.