Nemotron-3-Labs-Ultra-Math-RL: NVIDIA Quietly Open-Sourced the RL Checkpoint Behind Its IMO 2026 Run

NVIDIA dropped Nemotron-3-Labs-Ultra-Math-RL on Hugging Face on September 2–3, 2026, with no blog post, no X announcement and no press release — the model card simply appears, dated September 3, and explains itself in one line: it is the reinforcement-learning checkpoint, referred to as "Nemotron-3-Ultra-RL" in NVIDIA's accompanying technical report, that was "deployed as part of an ensemble system that achieved a gold-medal level score at the International Mathematical Olympiad 2026." That system's official score — 30 out of 42 points, above the 29-point gold threshold — was widely reported back in late July, when the base model's weights had been public since June. What was not downloadable then was the pair of post-trained specialists the competition run actually used. One of them is now sitting in a public Hugging Face repo with zero likes and a near-zero download count.

This is a quiet-release story, so it is worth being precise about what is knowable and what is not. Knowable from the repository and the report: the checkpoint's architecture, its training recipe, the role it played in the IMO 2026 submission, and the datasets and benchmark NVIDIA released alongside it. Not yet confirmed: any vendor announcement, any independent third-party benchmark of this checkpoint, and any hosted API — this is a self-host weights release under the OpenMDW-1.1 license, and running it means standing up roughly 1.5 TB of Blackwell-class HBM.

What actually shipped this week

The repository nvidia/Nemotron-3-Labs-Ultra-Math-RL was created on Hugging Face on September 2, 2026 (19:11 UTC) and last modified September 8. It is one item in a coordinated release collected under nvidia/nemotron-labs-imo-2026, which contains:

• The two post-trained competition checkpoints — Nemotron-3-Labs-Ultra-Math-RL (this subject) and its supervised-fine-tuned sibling Nemotron-3-Labs-Ultra-Math-SFT, both under OpenMDW-1.1.

• The two training corpora behind them — Nemotron-Math-Proofs-v3-SFT and Nemotron-Math-Proofs-v3-RL, both CC-BY-4.0.

• Nemotron-IMO-Bench, a new evaluation benchmark of 200 unpublished olympiad-level problems (50 algebra, 50 combinatorics, 50 geometry, 50 number theory), each paired with a human-curated reference proof, created with the mathematician Titu Andreescu specifically so the problems cannot appear in any training set.

• The base checkpoint NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 they are all finetuned from.

The collection description points to the technical report that ties it together: "An Open Recipe for IMO Gold: Training Nemotron for Olympiad Mathematics" (the PDF in NVIDIA's NeMo-Skills repo is dated September 8, 2026). Alongside the checkpoints NVIDIA released the inference pipeline, the submitted proofs, the RL training recipe, and a full compute accounting of the competition run. The framing is explicitly reproducible-science: this is NVIDIA saying, here is everything needed to rebuild the system.

What Nemotron-3-Labs-Ultra-Math-RL is

Architecturally this is not a new base model — it is a finetune of the general-availability NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16, the Mamba2–Transformer hybrid Latent Mixture-of-Experts checkpoint NVIDIA first shipped on June 4, 2026. The Math-RL checkpoint keeps that architecture and scale: 550B total parameters with 55B active, multi-token prediction, and support for context windows up to 1M tokens. What RL training changes is the specialization, and it is narrow in a deliberate way:

• Purpose — solve difficult competition math problems and act as a proof judge: the model is trained to produce a rigorous solution, self-evaluate it against a 0 / 0.5 / 1 rubric, and identify mistakes in proofs.

• It is not a general chatbot — the card and report describe a specialist worker for a proof-search pipeline, not an instruction-following assistant.

• License — OpenMDW-1.1, NVIDIA's permissive open-model license that allows commercial and non-commercial use, same as the base and the earlier Nemotron Labs teacher releases.

• Deployment — no hosted list price anywhere; you download safetensors and self-host. NVIDIA's reference config is a single node of 8× B200 (~1.5 TB aggregate HBM), with multi-node options across H100/H200/GB200/GB300. vLLM is the recommended runtime, with a reasoning parser, the Qwen3-Coder tool-call parser, Mamba SSM cache settings, and MTP speculative decoding on 5 tokens all called out in the card.

Why this checkpoint matters: it was inside the IMO 2026 system

The IMO 2026 result is the reason anyone is looking at this model, so the distinction that matters is: the 30/42 is a system score, not a single-model score. NVIDIA's submission was a two-stage, multi-checkpoint pipeline, and Nemotron-3-Labs-Ultra-Math-RL was a component with two jobs in it.

Per the report, the first stage ran an iterative generate–verify–refine search for up to eight rounds per problem. Round one sampled 384 proof attempts — 16 each from eight different solution-strategy prompts, from each of the three checkpoints: the general-availability model, the SFT specialist, and the RL specialist. Search-time verification used the RL and SFT checkpoints as a two-model panel: eight independent judgments from each, and a proof was accepted only if all 16 judgments scored it 1. A later high-compute stage re-graded every finalist with all three checkpoints (16 IMO-style judgments each, on a 0–7 scale) and selected the single submission per problem.

The official outcome, as reported in the paper and matching the late-July coverage: 30 out of 42 points on the IMO 2026 paper, above the 29-point gold-medal cutoff, with full marks on Problems 1, 2, 4 and 5 and one point each on Problems 3 and 6, graded by official IMO graders. NVIDIA's own resource accounting says all six submitted proofs were found within roughly 707M generated tokens and 1,464 GB200 GPU-hours; completing the in-flight rounds brought the full run to about 2.31B tokens and 4,800 GB200 GPU-hours.

Two internal numbers from the report give a sense of why the RL checkpoint earned its place in the ensemble. On NVIDIA's 30-problem development set, scored by an independent jury of GPT-5.5, Gemini 3.1 Pro and Claude Opus 4.8 following a MathArena-style procedure, the RL specialist was the best single-checkpoint pipeline end-to-end (180 of a possible 210 jury points, versus 165 for the SFT specialist and 162 for the base model), though the SFT checkpoint solved more problems in round one. And on a 300-proof audit set, the deployed RL+SFT verification panel cut the false-accept rate — the failure that ends search on an invalid proof — to about 1.1%, versus 12.4% for the RL checkpoint judging alone and 31.6% for the base model. The report's point is that the two selective specialists catch each other's errors under a unanimity rule.

These are NVIDIA's own ablations on NVIDIA's own development set, reported in NVIDIA's paper — worth treating as vendor-reported evidence of why the design works, not as independent scores. The development set itself is only 30 problems, and no outside lab has run this checkpoint yet.

The RL recipe, briefly

The training data and method are fully open, which is the actual news for anyone doing math-reasoning RL research. The high points from the report:

• Data — problems are drawn from the AoPS subset of Nemotron-Math-Proofs-v1, filtered to those the base Ultra model solves in one to three of four attempts (as judged by DeepSeek-V3.2-Speciale) — hard-but-tractable curriculum problems. That filter yields 9,597 proof-generation prompts, released as Nemotron-Math-Proofs-v3-RL.

• Reward — follows the DeepSeekMath-V2 design but drops the self-analysis reward term; the judge signal comes from a proof-judge service during training.

• Algorithm — asynchronous RL built on NeMo-RL, similar in spirit to PipelineRL: a fixed pool of prompts kept in flight, completed sequences fed to the trainer as batches fill, truncated importance sampling, and low-probability tokens masked on positive samples when entropy climbs. The config used a 131,072-token context and roughly 128 trainer nodes, 128 inference nodes and 16 judge nodes of 4× GB200 each.

The sibling SFT checkpoint, for contrast, was a long-context supervised fine-tune from the same base over a quality-filtered corpus of 414,890 proof, refinement, verification and meta-verification traces covering 15,818 problems, run on 512 GB200 GPUs.

What is not yet known

Honest sourcing cuts both ways here, and a reader deciding whether to care about this release should hold four caveats:

• No announcement. As of writing, NVIDIA has not formally announced the collection; it appeared on Hugging Face. The technical report PDF is dated September 8, so the written recipe is effectively being published this week too.

• No independent benchmarks. Every performance figure attached to this checkpoint — the dev-set ablations, the verifier audit, the IMO 2026 system score — originates from NVIDIA's own report. The IMO score itself has the strongest provenance of the set because it was graded under official competition conditions, but it is a system-level result, and the checkpoint's contribution to it is not something an outside lab has reproduced.

• No hosted API, no list price. This is a self-host release. Anyone wanting to call it needs the hardware. The "NVIDIA Nemotron 3 Ultra reached gold at IMO 2026" coverage from July can easily be misread as "download this and it solves olympiad problems" — the honest version is "download the components of the system that did."

• The gold framing. NVIDIA's own language is "reaching the gold-medal threshold," and the paper is careful that the model was part of an ensemble. Treat any claim that this single checkpoint alone is "gold-medal-level" as unsupported.

Who this is for

This release is aimed at a specific reader: a lab or researcher working on mathematical reasoning, proof generation, or RL with verifiable rewards, who wants a reference implementation of a system that cleared an olympiad gold threshold in natural language — no formal prover, no tools, no internet. For that reader the value is the whole package: weights, SFT and RL corpora, the NeMo-Gym-formatted prompts, the inference pipeline, the submitted proofs, and the compute accounting that says what a competition run actually costs in GPU-hours.

For everyone else, the practical note is that olympiad-grade proof search is overkill for most real math workloads. AIME- and contest-level problems, and essentially all applied math-in-code work, are handled comfortably by far smaller models — which matters because a 550B checkpoint is not something you spin up for a batch job. The smaller open math models and the frontier API models are reachable through a single API on OrcaRouter at the providers' list prices (no markup on our side, so a vendor price cut is live the same day), with automatic failover if you want to try an unproven model without betting a production path on it. Start there; go self-host a 550B only when the workload genuinely demands frontier-scale proof search.

The open question to watch is whether this quiet drop gets an official announcement and a formal release note, and whether anyone outside NVIDIA runs the recipe end-to-end and reports an independent IMO-Bench score. That benchmark — 200 fresh, unpublished olympiad problems — is arguably the most useful artifact in the collection: it is exactly the kind of contamination-resistant eval the field has been missing. If the recipe reproduces, this week's silent Hugging Face upload will turn out to be one of the more consequential open-model releases of the year. If it does not, the collection is still a detailed, honest account of what one frontier-scale run of generate–verify–refine actually required.