A generated title card for Nemotron-Labs-Sigma-distill-8B-nfe4, noting a 12 November 2026 release, 8 billion parameters, a fixed 4 denoising evaluations per 32-token block, five downloads and zero likes in the last month, and a footer stating that all figures come from NVIDIA's model card and are not independently verified.
Engineering & Research

Nemotron-Labs-Sigma-distill-8B-nfe4 Landed With No Announcement and Five Downloads

Author

jinhao.song

Date Published

Latest models · 20View all models →
Benchmarks: Artificial Analysis · updated daily
Back to all posts

Hugging Face counted five downloads of nvidia/Nemotron-Labs-Sigma-distill-8B-nfe4 in the last thirty days, and zero likes, when this piece was written on 17 November 2026. That is the whole reception so far for an 8-billion-parameter language model NVIDIA published six days earlier. There was no blog post, no build.nvidia.com listing, no launch thread, no benchmark table in a press release. The checkpoint's own card dates the release 11/12/2026; the repository's first commit is 9 November at 05:47 UTC, and the last edit to it was on 16 November, which rewrote the conversion to preserve fp32 geometry tensors. Everything below is read off that repository, its configuration files and the technical report it cites, and the parts that are not confirmed are labelled as such rather than filled in.

The repository is the only primary source, and it is unusually explicit

Most quiet releases leave a stub card. This one does not. It ships a provenance file, a checkpoint integrity manifest with a SHA-256, an integrity hash for the tokenizer, four Model Card++ subcards, and a generation config that pins the sampler. That level of bookkeeping is what you get when a model has been through an internal release process and then simply not been announced — the artifacts a launch post would summarise are all present, minus the post.

The paper behind it is real and dated: arXiv:2610.02665, Large Language Continuous Diffusion Models, a seventeen-author NVIDIA paper whose v2 landed on 12 November, the same day the card says the weights did. The paper describes Nemotron-Labs-Sigma as the first family of large-scale continuous diffusion language models at 3B and 8B. The card's evaluation section credits its numbers to "the 10-November restricted-release handoff's verified ten-seed, six-task statistics" — a sentence that tells you the file was being circulated privately for at least two days before the public repos were switched on.

Screenshot of the Hugging Face model card for nvidia/Nemotron-Labs-Sigma-distill-8B-nfe4, showing the model overview describing an 8-billion-parameter continuous diffusion language model distilled with Phased Distribution Matching Distillation, the native 4-NFE 32-token block generation contract, the NVIDIA Nemotron Open Model License, and the release date line reading Hugging Face 11/12/2026.

What is actually in the checkpoint

Strip the prose and the spec sheet is short. Every line below is from config.json, generation_config.json or the card:

• Backbone — a Ministral-style decoder-only transformer: 34 layers, hidden size 4,096, FFN 14,336, 32 query heads and 8 key-value heads at head dimension 128, SwiGLU, RoPE with a YaRN scaling block in the config.

• Generation head — a 16-dimensional continuous token-embedding diffusion head, with the auxiliary autoregressive readout head kept in the checkpoint rather than pruned.

• Parameter count — 8.0B by the card's naming, 9.03B by the safetensors census (9,026,818,048 BF16 tensors plus 2,097,186 F32), 18.06 GB across four shards. The gap between the two numbers is the auxiliary head.

• Decoding contract — 32-token blocks, exactly 4 denoising function evaluations plus one readout, so 5 denoiser calls per block, which is 6.4 generated tokens per forward pass.

• Sampler — the native fresh_x0 path with self-conditioning carried between steps, classifier-free guidance fixed at 1, score temperature fixed at 1.

• Context — 4,096 tokens for prompt plus generation. English only. No token-ID EOS stopping; you stop on strings or a token budget.

• Runtime — PyTorch 2.4 or newer, Transformers 4.55 or newer, trust_remote_code, a CUDA GPU in bfloat16. CPU is explicitly unsupported, and the card gives the reason: the sampler is numerically degraded on CPU and produces unusable output.

• Licence — the NVIDIA Nemotron Open Model License, which the card states permits commercial and non-commercial use. The card also states, in the use-case section, that the model runs offline on your hardware and that NVIDIA hosts no service.

That last line matters for anyone who assumed a Sigma endpoint exists somewhere. It does not. There is no Nemotron-Labs-Sigma entry on build.nvidia.com, and the model is absent from the OrcaRouter catalogue along with the rest of the Nemotron line — so there is nothing to call today, only something to download.

Four checkpoints, four different models

The naming makes the variants look like one model at four settings. They are not: each NFE value is a separately distilled student, and the card is blunt that the step count is baked in. "This variant is trained for NFE=4 only, and the bundled inference code intentionally does not expose an arbitrary step override" — because other step counts would evaluate phase endpoints the student never trained on. So you do not tune nfe4 down to two steps at inference; you download nfe2.

The published six-task means, all ten-seed with two standard errors, make the trade-off legible for the first time in this family:

• NFE 2 — GSM8K 80.17, MATH 55.72, HumanEval 48.17, MBPP 37.52; six-task mean 53.02. 3 denoiser calls per block, 10.7 tokens per call.

• NFE 4 — GSM8K 86.86, MATH 61.52, HumanEval 60.61, MBPP 57.50; six-task mean 66.36. 5 calls per block, 6.4 tokens per call.

• NFE 8 — GSM8K 88.44, MATH 64.40, HumanEval 65.18, MBPP 62.60; six-task mean 70.34. 9 calls per block, 3.6 tokens per call.

• NFE 16 — GSM8K 89.61, HumanEval 67.68, MBPP 66.20, three-task mean 74.50. 17 calls per block, 1.9 tokens per call. And a caveat the other three do not carry: this checkpoint is marked pending_ten_seed_six_task_validation, its figures are a single run at seed 42 on three tasks, and the card says to treat them as provisional and not directly comparable.

The shape of that ladder is the interesting part. Doubling steps from 2 to 4 buys 13.3 points of six-task mean. Doubling again to 8 buys 4.0. Going to 16 buys a further ~4 on the three tasks that were run — from a different, weaker evaluation. In other words the distillation curve flattens hard right where the model becomes competitive, and the nfe4 checkpoint sits just past the knee.

A generated bar chart of the Nemotron-Labs-Sigma-distill-8B checkpoints, plotting six-task mean accuracy against denoising evaluations per 32-token block: NFE 2 at 53.02, NFE 4 at 66.36 and NFE 8 at 70.34 as ten-seed figures, with NFE 16 shown separately at a three-task mean of 74.50 and marked as a single seed-42 run pending validation.

Where the numbers come from, and where they stop

Every figure above is NVIDIA's, produced by NVIDIA, on NVIDIA's own evaluation harness, and reported with a seed count that makes it look more independent than it is. Ten seeds with two standard errors is a serious protocol — it tells you the nfe4 HumanEval score of 60.61 has a ±1.43 interval, which is wide enough that you should not read a two-point gap against anything as real. But ten seeds of the same harness is not ten laboratories.

What does not exist as of 17 November: an Artificial Analysis page for any Sigma model, any entry in a public model-price catalogue, any vLLM or SGLang support, and any third-party reproduction. The check is quick and it comes back empty. vLLM's model registry on main has no nemotron_labs_sigma architecture; the SGLang pull requests that mention Nemotron diffusion are for the older Nemotron-Labs-Diffusion-8B lineage and are still open. Transformers has no native nemotron_labs_sigma module, which is why the card tells you to load with trust_remote_code and ship the repository's 40 KB modelling file along with the weights.

So the honest status is: a released checkpoint with a published evaluation protocol and zero external verification. That is the normal state for a diffusion language model in November 2026, and it is not a criticism of the model — it is a description of how much of the card you can plan against.

Running it is a Python script, not a serving stack

The card's quickstart is eleven lines: load the tokenizer and model with remote code, cast to bfloat16, call generate with a max token count and optional stop strings, and read back the NFE count the model reports. There is no chat template in this repository, no system prompt, no tool calling, no JSON mode — the card says plainly that this is benchmark-style prompting and that the model is the distilled student of a base checkpoint, not an assistant. If you want conversation, the collection carries an instruct variant of the parent at 8,192 tokens of context.

Two operational details will bite anyone who skims. First, stopping: there is no token-ID EOS stop, so a request that never produces your stop string runs to the token budget, and the budget is shared with the prompt inside a 4,096-token window. Second, memory: 18 GB of bfloat16 weights before any KV cache, on a model whose only supported path is PyTorch eager code from the repository itself — there is no paged attention, no continuous batching, no quantised build in the collection. A single request on one GPU is the supported configuration today.

Throughput is the number people will want and the card does not give it for this checkpoint. The parent model's card reports roughly 120–140 tokens per second for 256 generated tokens on one H100 using the DDPM sampler at 32 steps per 64-token block. The distilled nfe4 does 5 calls per 32 tokens instead of 33 calls per 64, which by call count is a little over four times fewer forward passes per token — but that is arithmetic on step counts, not a measurement of this checkpoint, and the card publishes no tokens-per-second figure for it.

What is not confirmed

Three things a launch post would have settled, and this release does not:

• Whether NVIDIA intends this as a product or a research artifact. The licence permits commercial use and the card is written in product language, but there is no hosted endpoint, no support statement and no mention in any NVIDIA blog post through 16 November.

• What the teacher is worth. The card says the student was distilled from an "MS-SUSReg-8B" teacher of the same architecture family, and the provenance file names the run 8b_ms_susreg_n4_20261109 at student step 1200. The released base Nemotron-Labs-Sigma-8B is described as the same lineage — but the base card publishes no numbers at all, only a pointer to the paper, so the distillation loss cannot be read off the two cards side by side.

Screenshot of the nvidia/nemotron-labs-sigma collection page on Hugging Face, listing fourteen repositories: the 3B and 8B base and instruct models, distill variants at NFE 2, 4, 8 and 16 for both sizes, and self-speculative distill checkpoints for 3B and 8B, with the collection description reading continuous diffusion language models, base, instruct, few-step distilled and self-speculative variants.

• Whether the nfe16 checkpoint is the best of the family or an artifact of an unfinished evaluation. Its numbers are the highest on the card and the least trustworthy, and the card says so itself, which is more candour than most release notes manage.

Why the quiet release is the story

NVIDIA has shipped a lot this autumn — Bolt, Super VL, Transcribe, the Dory line, all with blog posts and build.nvidia.com pages. Sigma arrives with none of that, from the same organisation, with a paper, a licence, four Model Card++ subcards and a five-download counter. The likeliest reading is that this is a research release deliberately kept out of the product surface: the card's intended use is "latency-sensitive generation research, speculative and parallel decoding studies," which is the language of a lab artifact.

For a reader deciding whether to care, the practical question is not whether the model is good in the abstract — it is whether a 4-NFE continuous diffusion LM at 4K context with no serving stack beats the autoregressive model already in your pipeline. Answering that requires running it, and the arithmetic that makes it worth the GPU hour is simple: 6.4 tokens per forward pass, against one for a standard decoder. Whether that survives contact with your workload is exactly the thing the missing independent benchmark would have told you.

OrcaRouter does not serve this checkpoint, and it is not going to appear behind a per-token price until someone builds a serving path for a block-diffusion architecture with a fixed step count — the cost model does not map onto input and output tokens the way an autoregressive API does. What is live at OrcaRouter is the comparison set: 200-plus autoregressive models behind one key at provider list price with no markup, so the Sigma question can be answered against a real baseline rather than against a card. Routing both through one endpoint is also the cheapest way to keep the experiment off your production path while you decide.