Nemotron 3.5 Bolt 24GB: What NVIDIA Actually Put in a 24GB File

The headline number is not the parameter count. It is the byte count. On 15 October 2026 NVIDIA released NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB, a vision-language agent model whose 74 billion parameters occupy 23.75GB in the format most people will actually download — a 22.02GB GGUF plus a 1.73GB vision projector file. Both figures come from the published repository's own file listing, not from the blog post, and they are the reason the name ends in a number. This is what that number buys, what it hides, and what it does not include.

What shipped, and when

NVIDIA's newsroom framed 15 October as a family expansion rather than a single-model launch: Nemotron 3.5 Super VL, a 120-billion-parameter multimodal model at 12 billion active parameters, arrived alongside the Bolt checkpoint, and a much smaller sibling, Nemotron 3.5 Bolt E4B 4GB, is dated for 23 October. Bolt joins Nemotron 3.5 Lightning (August) and Nemotron 3 Ultra in a family that now covers data-centre reasoning, high-throughput serving and local edge agents.

The release is genuinely new, and it is also slightly older than the announcement. The Hugging Face repository nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB was created on 7 October 2026; the model card inside it carries a release date of 15 October, the same date NVIDIA's own research page is stamped "Published". Eight days of files sitting in public view before a single word of explanation is a normal pattern for a vendor shipping under embargo coordination with hardware — RTX Spark laptops are dated 16 October, the day after the announcement — and it is worth knowing which half of this model's public record came from a marketing decision and which came from a file listing.

How 74 billion parameters fit in 24 gigabytes

Two mechanisms, and they are separable.

The first is structural. Bolt is not a smaller model trained from scratch; it is the Super 3.5 VL parent compressed by an algorithm NVIDIA calls Star Elastic, a router-guided architecture search that had to satisfy one constraint — a deployment footprint below 24GB at batch size 1 and 128K context — while losing as little knowledge-distillation accuracy as possible. What it produced is a 74B total / 9B active mixture-of-experts hybrid, with the expert top-K fixed at 9 where the parent used 22. NVIDIA's own framing of the result is blunt and checkable: for a fixed memory budget, compressing a large model beats training a small one.

The second is precision, and it is where most of the savings live.

• Expert weights — routed and shared FC1 and FC2 layers quantised to IQ2_XXS at 2.0625 bits per element, which is the bulk of the model by parameter count.

• Mamba layers — NVFP4, a 4-bit floating-point format, and only on the layers where the extra precision is worth more bytes.

• Embeddings, output head, attention and the latent MoE weights — Q8_0.

• Recovery — quantisation-aware distillation rather than a post-hoc round-to-nearest pass, which is the difference between a compressed model and a broken one.

Read those together and the "24GB" stops being a marketing summary and becomes a budget allocation. The 2-bit experts are cheap per byte and numerous; the layers that matter for stability keep 4 or 8 bits.

The 24GB figure, checked against the files

The claim survives an audit, which is not always true.

• GGUF weights — 22,018,326,528 bytes, or 22.02GB decimal, in a single file.

• Vision projector — 1,733,692,736 bytes, or 1.73GB, shipped separately as mmproj, which is how llama.cpp takes image and video input.

• Safetensors build — the same model in vLLM-ready shards totals 24.12GB across 90 files, marginally over the headline.

• BF16 reference — 149.8GB across 63 shards, published separately for anyone who wants to fine-tune rather than infer.

Two honest caveats sit on top of that. First, weights are not a running model. The vLLM recipe NVIDIA publishes pins the KV cache to fp8 and sets the served context to 262,144 tokens with max-num-seqs 1 — a single concurrent sequence. That is the difference between fitting in 24GB and running fast in 24GB, and it is the kind of detail the memory-size headline never carries. Second, 24GB is the number the architecture search was given, not a number measured on your machine: driver overhead, the operating system and whatever else is on the laptop are all outside it, which is precisely why the pitch is "run alongside your apps" rather than "run instead of them".

The speed claim, and the machine it was measured on

NVIDIA designed for at least 80 tokens per second and reports 83.99 on AIPerf SPEED-Bench-Coding. That number needs three pieces of context before it means anything: it was produced with llama.cpp on Windows 11 ARM x64, on a Surface Laptop Ultra with 128GB of memory, with the LiLiCorr speculative drafter attached and seven drafted tokens per step. The drafter is not optional garnish — it is a 1.13B companion checkpoint whose whole purpose is to raise acceptance length (NVIDIA reports means of 4.68 under greedy decoding and 4.25 with sampling at concurrency 64 in vLLM). Strip the drafter out and you are measuring a different model.

The second speed figure is the more interesting one, because it is a completion time rather than a rate: 600.80 seconds of replay duration on AA-AgentPerf Local, against 453.97 for Qwen3.6 35B-A3B, 480.68 for Gemma 4 26B-A4B and 1,043.28 for Qwen3.8 27B — all from NVIDIA's own table, all on the same hardware. Bolt wins that row by finishing the same agent trajectories faster than a smaller model, and loses the throughput row to both of the others at 83.99 against 96.63 and 100.56 tokens per second. Fast and quick are not the same measurement, and this is a model that trades the second for the first.

What the independent measurement says

One number in the public record about Bolt was not produced by NVIDIA: the Artificial Analysis Intelligence Index scores it at 21, which puts it fourth of 67 models in its class. That panel also reports no speed and no cost for it, because there is no public API to call — and it lists the model's input modality as text and image, not video, which is a narrower reading than the vendor's.

The same page notes the model generated 230M tokens while being evaluated, which it labels very verbose against a median of 100M. Reasoning that produces more tokens on the same task is not automatically worse, but it costs something on a local device where every token is generated by a fan.

Everything else in circulation is vendor-reported. NVIDIA's own thirteen-benchmark table, in which Bolt leads its memory class on PinchBench and sits second on WebArena and BrowseComp, compares against models chosen partly for fitting in 24GB at 4-bit — a peer group defined by a constraint Bolt was engineered to satisfy. It is not dishonest; it is also not a ranking of agent models in general, and the one cell that goes the other way is instructive: at 16.90 on AA-Omniscience accuracy, Bolt trails Gemma 4 26B-A4B at 22.17 in the same table.

What is not in the box

There is no endpoint. No metered rate, no provider serving it, no way to evaluate it on your own traffic without completing the deployment. What there is instead is runtime support that landed at release rather than months later: llama.cpp and the GGUF checkpoint, Ollama, LM Studio, Unsloth for fine-tuning, and vLLM for the quantised checkpoint on DGX Spark and the BF16 checkpoint elsewhere. NVIDIA's own blog lists those five by name, and the breadth is the real product here — a 24GB file is only useful if the thing that loads it already exists on your machine.

Hardware reach is wider than the launch-day story about RTX Spark laptops suggests: GeForce RTX PCs, DGX Spark, DGX Station and RTX PRO workstations are all named, and the model card lists Ampere, Hopper and Blackwell for the vLLM path.

The licence is OpenMDW-1.1, which is permissive-enough to build on and unfamiliar-enough that legal review is a real step rather than a formality — worth knowing before an architecture decision depends on it.

A copy-paste slip in NVIDIA's own technical blog puts the checkpoint list under the heading "Nemotron 3.5 Super VL 74B-A9B 24GB" rather than Bolt. It is a typo in a fast ship, not a fork in the family, but it is a useful reminder that vendor pages written on announcement day are drafts with a publication date on them.

For teams who also call models they do not own

The locally-run half of an agent stack and the hosted half are usually the same project, and Bolt only addresses one of them. OrcaRouter does not host any Nemotron checkpoint — that was checked against our own catalogue rather than assumed — so nothing here is a pitch for a model we serve. What we do is the other side of a mixed deployment: one key across 200-plus models at the provider's list price with no markup added, automatic failover when a provider wobbles mid-request, and a routing DSL for describing a fallback chain as configuration instead of application code. If Bolt runs the local half, that is the half that never appears on a bill; the routing question is what happens to the rest.

What to watch

Three things will settle this model's reputation, and none of them are in the announcement.

The first is independent latency on real hardware. NVIDIA's 83.99 tokens per second is a controlled benchmark with a speculative drafter attached; someone will eventually publish what it does on a laptop running a browser and an editor at the same time, and that number will matter more than this one. The second is the E4B variant on 23 October, which tests whether the compression recipe scales down or only works at one size. The third is whether anyone outside NVIDIA post-trains the BF16 checkpoint — because a 24GB deployment artifact with an open licence and a 150GB customization artifact next to it is a deliberate two-track release, and the second track is the one that decides whether this becomes a platform or a checkpoint.

For now the fact stands on its own. A 74-billion-parameter multimodal agent fits in the memory of a laptop that costs less than a workstation, runs at roughly 84 tokens per second with a drafter attached, scores 21 on the one independent index that has measured it, and cannot be called as an API by anyone. That is a specific and unusual combination, and it is worth being precise about which half of it you need.

None of that is hypothetical: the documented AI failures and agent incidents behind releases like this one are collected in the OrcaRouter AI Incident Archive, where each entry carries its own sources, severity, confidence grade and disputed flag.