Nemotron 3.5 Bolt is uit: wat NVIDIA's 24GB 2-bit checkpoint werkelijk bevat

The repository for N​VIDIA N​emotron 3.5 Bolt 74B-A9B 24GB appeared on Hugging Face on 7 October 2026 and sat there, quietly, for eight days. On 15 October N​VIDIA announced it — a blog post, a newsroom release and a research page stamped with that date — which means the interesting part of this story is not the launch. It is the gap. For eight days the only thing anyone could know about N​emotron 3.5 Bolt was what a model card and a set of safetensors said, and everything else was inference. That gap has now closed, and it is worth being precise about what closing it actually settled, because a good deal of what people are repeating about this model came from the repository rather than from the announcement, and the two are not the same document.

What is certain now, in one paragraph, is this. N​emotron 3.5 Bolt 74B-A9B 24GB is a 74-billion-parameter mixture-of-experts vision-language model with 9 billion parameters active per token, released on 15 October 2026 under the OpenMDW-1.1 licence, storing its expert weights at 2.0625 bits per element so the whole thing fits in 24GB. It is designed to run local agentic work — tool calls, browsing, terminal use — on RTX and DGX Spark hardware. It is on Ollama as a 25GB tag with a 256K context window. It is not on any hosted API, and has no price. One part of that sentence is measured independently and the rest is the vendor's own word.

Twee datums, en welke je moet citeren

Er is een wezenlijk verschil tussen de twee datums, en dat bepaalt hoe je de rest van dit artikel moet lezen.

The Hugging Face repository was created on 7 October 2026 at 17:50 UTC. The model card inside it carries a Release Date field reading Hugging Face 10/15/2026. N​VIDIA's own research page for the checkpoint is stamped Published: October 15, 2026. The vendor blog post announcing the Bolt tier is dated 15 October 2026 and says, in as many words, that Super VL and Bolt 74B-A9B "are available now".

So the honest reading is that 15 October is the release date and 7 October is when the folder showed up. Anyone quoting 7 October as the launch is quoting a file timestamp; anyone quoting 15 October is quoting the vendor. The eight-day gap matters for a different reason than dates, though. It means that for over a week the model existed in a state where the repository was the only surface — no announcement, no pricing page, no leaderboard entry, nobody from N​VIDIA saying what it was for. Every early description of Bolt written in that window, including the ones now circulating, was built from the card. The card turns out to have been accurate. That is worth saying plainly, because the usual outcome of these situations is that it is not.

Wat alleen de repository bewijst

Everything in this section was readable from the model card and the config files before N​VIDIA said a word, and it is all still true.

• Parameters — 74 miljard totaal, 9 miljard actief per token. Sparse mixture-of-experts, geen dense model.

Architectuur — een hybride van Mamba-2 en Transformer met een LatentMoE-backbone en een C-RADIO v4-H vision-encoder. De configuratie registreert de klasse als NemotronH_Omni_Reasoning_V3.

• Opgeslagen precisie — IQ2_XXS met 2,0625 bits per element op de FC1- en FC2-lagen van gerouteerde en gedeelde experts, waar het grootste deel van de gewichten zit. De Mamba-lineaire lagen staan op NVFP4, en de embeddings, output head, attention en LatentMoE-gewichten op Q8_0.

• Context — tot 256K tokens, of 262.144 in de deploymentrecepten.

• Invoer en uitvoer — tekst, afbeeldingen en video in; tekst uit.

• Licentie — OpenMDW-1.1, en de kaart zegt het duidelijk: "geschikt voor commercieel of niet-commercieel gebruik."

• Bases and provenance — the card declares its base model as the BF16 checkpoint of the same name and its relation to it as quantized, and the dataset tags point at N​VIDIA's N​emotron pre-training and post-training collections.

• Modeldatums — december 2025 tot mei 2026, met een afsluitdatum van de post-trainingdata in mei 2026. Lees dat goed als actualiteit voor jou belangrijk is: het model is een release uit 2026, gebouwd op data die viereneenhalve maand voordat het werd uitgebracht, stoppen.

• Runtimes — vLLM on N​VIDIA Ampere, Hopper and Blackwell, and llama.cpp for RTX Spark and DGX Spark. Ollama's listing is more blunt about the limits: CUDA deployment is documented, and Apple Silicon and Metal are not currently supported.

Wat de aankondiging toevoegde

Het leveranciersmateriaal voegt drie dingen toe die de repository u niet had kunnen vertellen.

First, the family framing. Bolt is positioned as the edge tier of the N​emotron family, below N​emotron 3.5 Super VL at 120B for always-on server agents and above nothing in particular — N​VIDIA's own sentence sorts them: Super VL and Bolt "cover the full range of agent deployments, from large enterprise servers down to edge devices and local hardware." N​emotron 3.5 Bolt E4B 4GB, a 4-billion-parameter model for conversational and physical agents, arrives on 23 October.

Second, the compression story, which is the genuinely novel engineering claim here. Bolt is not a model that was trained small and then quantised. It is N​emotron 3.5 Super VL — 120B total, 12B active — put through Star Elastic, N​VIDIA's router-guided architecture search, under a hard constraint of fitting below 24GB of memory at batch size one and 128K of context. Expert top-K came down from 22 in the parent to 9 in Bolt, the MoE intermediate widths were searched per layer, and then the compressed network was quantised and put through quantisation-aware distillation to claw accuracy back. N​VIDIA's stated conclusion is the interesting part and it is a real position, not a boast: for a fixed memory budget, compressing a large model down beats training a small one from scratch. That is a claim you can test, and nobody outside N​VIDIA has.

Third, two leaderboard placements stated by the vendor: #1 open model in its class on PinchBench, and #2 open model on BrowseComp. Both are N​VIDIA's characterisation of its own model against a class it defined.

What the announcement does not add is a served endpoint. There is no hosted API, no per-token price, and no way to call N​emotron 3.5 Bolt without acquiring the hardware it runs on. Artificial Analysis, which has scored the model, lists its speed and cost fields as unavailable for exactly that reason: no publicly available API to measure.

The part nobody outside N​VIDIA has run

Hier moet het artikel vaart minderen, want de kloof tussen wat is gemeten en wat is beweerd is ongewoon groot voor een model van deze omvang.

On the measured side there is one number, and it is a good one: Artificial Analysis puts N​emotron 3.5 Bolt 74B-A9B at 21 on its Intelligence Index, which places it fourth out of 67 models in its comparison class. That is an independent measurement of the actual 2-bit checkpoint, not of a BF16 reference build, and it is the single most useful figure available on this model today.

On the asserted side there is everything else. The model card carries a thirteen-benchmark table with four columns — Qwen3.6-35B-A3B in BF16, Gemma 4 26B-A4B in BF16, N​emotron 3.5 Lightning 30B-A3B in BF16, and Bolt in IQ2 — and every row of it was produced by N​VIDIA, including the three baseline columns, which N​VIDIA selected because they fit in 24GB with 4-bit quantisation. Bolt is the bolded winner in ten rows and is not the winner in three: AA-Omniscience accuracy, where Gemma 4 26B-A4B scores 22.17 against Bolt's 16.90; AA-Omniscience non-hallucination, where N​emotron 3.5 Lightning 30B-A3B scores 68.81 against Bolt's 61.27; and PinchBench, where Qwen3.6-35B-A3B scores 88.07 against Bolt's 87.82. Publishing the three rows you lose is a point in the vendor's favour and does not change what the table is: unreproduced.

The speed numbers deserve their own paragraph because they cut against the marketing. N​VIDIA's own measurement table, taken with llama.cpp on Windows 11 ARM64 on a Surface Laptop Ultra with 128GB, shows Bolt at 83.99 tokens per second on AIPerf SPEED-Bench-Coding — behind Qwen3.6-35B-A3B at 96.63 and Gemma 4 26B-A4B at 100.56. On the same page, end-to-end replay duration across realistic agent trajectories is 600.80 seconds for Bolt against 453.97 for Qwen3.6 and 480.68 for G​emma. The pitch is intelligence per gigabyte, not speed, and the vendor's own table is consistent with that pitch rather than with a claim of being fastest.

So the shape of the evidence is: one independent composite score that is respectable, one vendor benchmark table nobody has reproduced, one vendor speed table on a laptop that nobody outside N​VIDIA has run, and zero independent latency measurements. Treat any article that reports Bolt's PinchBench figure without saying who produced it as unreliable, including any article that agrees with this one.

Wat kwam er samen mee?

Een modelrelease bestaat zelden alleen uit de gewichten, en de ondersteunende cast van Bolt is groter dan gebruikelijk voor iets zonder API.

The LiLiCorr speculative drafter ships with it — 1.13 billion parameters excluding the shared embeddings and output head, trained jointly with the target, and N​VIDIA reports mean acceptance lengths of 4.68 under greedy decoding and 4.25 under probabilistic sampling with seven drafted tokens at concurrency 64 in vLLM. The drafter is a separate downloadable checkpoint, not a runtime flag, and it is the mechanism behind the 84 TPS figure quoted for RTX Spark.

Framework support landed fast. Ollama lists nemotron-3.5-bolt in three tags — a default, a 74B-A9B tag and an IQ2 tag — each at 25GB with a 256K context and text plus image input, with one-line launchers for C​laude Code, OpenCode, Hermes Agent and OpenClaw. Beyond that there is a GGUF build from N​VIDIA, a chain of community quantisations at 4-bit and 8-bit, and MLX conversions that appeared within a day of the announcement. For a checkpoint that runs on a single 24GB device, that ecosystem arriving in forty-eight hours is the real signal that this is a usable artifact rather than a demonstrator.

De praktische waarschuwing voor iedereen die het van plan is te gebruiken: het getal van 24GB is een ondergrens voor de gewichten, niet een bovengrens voor de werklast. Afbeeldingen, videoframes, gespreksgeschiedenis en toolresultaten delen allemaal het contextbudget, en Ollama's eigen opmerking zegt dat de bruikbare context afhangt van het beschikbare geheugen. Een 25GB-download die comfortabel past is niet hetzelfde als een 256K-token-sessie die comfortabel past.

Waar dit je brengt, en waar het je niet brengt

If your workload is local and agentic — terminal use, browsing, tool calling, code debugging on a device you own — N​emotron 3.5 Bolt 74B-A9B 24GB is now a defensible choice with one independent score and a set of weights you can inspect. The compression argument behind it is genuinely interesting, the licence permits commercial use, and the tooling showed up within a day.

If your workload needs to be called from code running somewhere you do not own, this model is not an option today, and the honest answer to "how do I use it" is that you run it yourself on N​VIDIA hardware or you wait. We checked our own catalogue rather than assuming, and there is no N​VIDIA model on it — so there is nothing here for us to route, and saying otherwise would be a lie. What OrcaRouter does offer is the other half of a mixed deployment: the models you would call when a request is too hard, too long, or too unfamiliar for a 2-bit local checkpoint, behind one key across 200-plus models at the provider's list price with no markup added, with automatic failover when a provider wobbles. Pairing a local Bolt with a routed API is a sensible architecture. Pretending Bolt is on the routed side is not.

The thing to watch is not the next benchmark table. It is whether anyone outside N​VIDIA publishes a latency run on the 24GB checkpoint on hardware a reader actually owns, and whether a hosted endpoint appears at all. Until one of those happens, the specification is a promise and one index score is the only thing standing behind it.

Een resultaat zoals dit is ook een herinnering eraan dat modelfouten al gedocumenteerd zijn: de AI-incidenten en agentfouten achter modellen in deze klasse zijn verzameld in het OrcaRouter AI Incident Archive, waar elke vermelding zijn eigen bronnen, ernst, zekerheidsgraad en betwistingsmarkering bevat.