Nemotron 3.5 Bolt 74B vs Nemotron 3.5 Lightning 30B-A3B: Same Recipe, Two Memory Ceilings

This is the only comparison on the N​VIDIA N​emotron 3.5 family tree where both sides were built by the same team for the same job at two different sizes, and the vendor has already run the head-to-head itself. N​VIDIA N​emotron 3.5 Bolt 74B-A9B 24GB and N​VIDIA N​emotron 3.5 Lightning 30B-A3B are both Mamba-2 hybrid mixture-of-experts checkpoints with speculative-decoding heads, both released under OpenMDW-1.1 for commercial use, both intended to be run by the person reading about them rather than rented from a data centre. Lightning shipped on 11 August 2026. Bolt's weights went up on 7 October 2026 and N​VIDIA announced it on 15 October.

The difference is a memory ceiling: 24GB for one, roughly 16GB for the other in its quantised form, and the eight points of Artificial Analysis Intelligence Index between them are what that ceiling buys.

One recipe, run at two sizes

Lightning arrived first, and it established the shape the whole 3.5 generation follows: a hybrid stack that interleaves Mamba-2 state-space blocks with mixture-of-experts and attention layers, Multi-Token Prediction heads so the model can draft several tokens at once and verify them in parallel, and separate companion drafters — DSpark for low-concurrency data-centre deployments, DFlash, and later the LiLiCorr drafter — that make speculative decoding a supported configuration rather than a research trick. Distillation from the N​emotron 3 Ultra teacher is the other half of the recipe.

Bolt is the same recipe pointed at a much tighter budget. Its 74B parameters are compressed to 24GB by Star Elastic, N​VIDIA's router-guided architecture search, which takes a parent checkpoint and searches for the per-layer expert count and MoE intermediate width that satisfies a hard deployment footprint while minimising knowledge-distillation loss. In Bolt's case the search was bounded at below 24GB at batch size 1 and 128K context, expert top-K came down from 22 in the parent to 9, and the expert weights were then quantised to IQ2-XXS at 2.0625 bits per element with quantization-aware distillation to recover accuracy. Mamba layers sit at NVFP4, embeddings and output layers at Q8_0.

The parent for Bolt is not Lightning. It is N​emotron 3.5 Super VL 120B-A12B, the 120-billion-parameter multimodal model N​VIDIA launched on the same day, which is why Bolt ships with a C-RADIO v4-H vision encoder and Lightning does not.

The spec contrast, both sides on one line

• Total and active parameters — Bolt: 74B total, 9B active per token. Lightning: 30B total, roughly 3B active per token.

• Architecture — Bolt: Mamba-2–Transformer hybrid LatentMoE plus a C-RADIO v4-H vision encoder. Lightning: Mamba-2 hybrid MoE with attention, no vision tower.

• Precision and footprint — Bolt: IQ2 at 2.0625 bits per element, packed to below 24GB. Lightning: BF16, NVFP4 and other quantised builds published separately.

• Context — Bolt: up to 256,000 tokens, with individual deployment recipes free to use less. Lightning: up to 1,000,000 tokens in principle, with roughly 256,000 the practical ceiling on a single H100.

• Inputs and outputs — Bolt: text, images and video in, text out. Lightning: text in, text out.

• Licence — OpenMDW-1.1 for both, cleared by N​VIDIA for commercial and non-commercial use.

• Speculative decoding — Bolt: the LiLiCorr drafter via DFlash, seven speculative tokens in the reference recipe. Lightning: MTP heads plus DFlash and DSpark drafters.

• Intelligence Index — Bolt: 21.4, fourth of 67 in its size class, independently measured by Artificial Analysis. Lightning: 12.9, independently measured.

• Price — Bolt: no metered rate anywhere; Artificial Analysis lists no input or output price and there is no published hosted endpoint. Lightning: $0.06 per million input tokens and $0.20 per million output as tracked by Artificial Analysis, with cached input around $0.006.

• Output speed — Bolt: no independent measurement published. Lightning: 305 output tokens per second with 0.57 seconds to first token, both independently measured, which is among the fastest figures on the panel's board.

The vendor's own head-to-head, read carefully

N​VIDIA's model card for Bolt publishes a table in which Lightning is one of three baselines, and it is worth reading as an argument rather than a scoreboard, because the company selected benchmarks where its new model wins.

• Terminal-Bench 2.1 — Bolt 50.98 against Lightning's 23.46. Both vendor-run, both on the BF16 Lightning build.

• GPQA Diamond — Bolt 86.43 against Lightning's 75.57.

• BrowseComp — Bolt 66.75 against Lightning's 36.81.

• HLE with no tools — Bolt 25.30 against Lightning's 10.47.

• PinchBench — Bolt 87.82 against Lightning's 83.43, a much narrower margin than the others and consistent with N​VIDIA's claim that Bolt is the number one open model in its class on that benchmark.

• AA-Omniscience non-hallucination — Bolt 61.27 against Lightning's 68.81. Lightning wins this one, and it is the row N​VIDIA chose to publish anyway, which is a point in the table's favour.

• AA-Omniscience accuracy — Bolt 16.90 against Lightning's 16.63. Effectively a tie, and the weakest pair of numbers on the table in absolute terms.

Every figure in that list is vendor-reported on the model card and unreproduced. Artificial Analysis has since measured the two index scores independently — 21.4 and 12.9 — and the direction matches, which is the most that can be said today.

Where the smaller model is the better answer

The index gap is real, and it is not the only axis. Lightning wins three rows that matter for specific deployments.

It is faster by a factor no one has measured on the Bolt side. 305 output tokens per second with sub-second time to first token is a different class of responsiveness from anything an unmeasured 2-bit 74B checkpoint can currently be assumed to reach, and it is measured. Bolt's own published throughput, 83.99 tokens per second on AIPerf SPEED-Bench-Coding with the LiLiCorr drafter active, comes from N​VIDIA's research page on a laptop-class machine and is not comparable to the panel's rig.

It has four times the context. A million tokens against 256,000, and the difference shows up in the independent long-context retrieval score: Lightning posts 49.19 on AA-LCR in N​VIDIA's table, Bolt 64.06 — Bolt is better at using long context, but Lightning is the only one of the two that advertises a million-token window, so if the workload is "read the whole repository" the ceiling may matter more than the retrieval quality inside it.

It is the only one of the two with a price. Lightning is available at $0.06 in and $0.20 out per million tokens, which makes it callable infrastructure rather than a hardware project. Nobody has published a hosted rate for Bolt, so the only way to run it is to own the machine.

What Lightning cannot do is see. It is text-only, and Bolt accepts images and video through the C-RADIO v4-H encoder, scoring 50.86 on WebArena vision where Lightning has no entry. If the agent reads screenshots, opens PDFs or drives a browser, that row alone decides the choice — and it is the row that comes from Bolt's Super VL parentage rather than from its parameter count.

What it costs to actually run Bolt

N​VIDIA's reference serving recipe for the 24GB checkpoint is not a single command you paste. It pins a specific vLLM nightly image, installs the b12x kernel package and downloads the kernels, fetches a reasoning-parser plugin file separately, and then sets the v2 model runner, the linear and MoE backends to b12x, the attention and Mamba backends to FlashInfer, explicit Mamba cache dtypes, a chunked-prefill flag and a speculative-decoding config pointing at the LiLiCorr drafter with seven draft tokens. The tool-call and reasoning parsers are named explicitly, and the media limits cap a request at one image and one video with a 0.5 video pruning rate.

Lightning's deployment story is a Docker container, an O​penAI-compatible endpoint and a model name you can also get from a hosted provider at six cents per million tokens. In practice, that difference is the whole comparison: Bolt is a weekend of work and a machine you own, Lightning is an afternoon and an invoice.

How to choose, and what would settle it

Take Lightning if your workload is text, your latency budget is tight, and either your context needs are large or you would rather pay per token than buy hardware. Its independent profile — fastest measured output speed in its class, sub-second first token, a published price — makes it the lowest-risk option in the family, and the lower intelligence score is priced accordingly. It is also the right default if you plan to pair a small local executor with a bigger hosted planner, which is exactly how N​VIDIA positions it.

Take Bolt if your agent touches pixels or runs without a network. The vision row is exclusive to it, the 74B-scale reasoning is eight index points ahead, and the 24GB envelope is the only one in the family that fits a 74B teacher-distilled checkpoint onto a single consumer-class accelerator. Accept that you are running the GPU-hosted planner locally and paying for it in latency you cannot yet predict.

For the hosted half of either architecture, OrcaRouter puts the models we do route behind a single key at the provider's list price with 0 percent markup added, and automatic failover across providers means a capacity problem upstream is retried rather than surfaced. We do not host any N​emotron model — the catalogue was checked rather than assumed, and the family is not in it — so the local side of this comparison is unaffected by anything we do. What a routed setup buys is a plan B for the planner while you evaluate, which matters here because neither of these two checkpoints has a published independent latency profile you can size a production path against.

What would settle the choice is one number that does not exist yet: an independent throughput and latency run on the 24GB checkpoint at batch size one. Until somebody publishes it, the case for Bolt rests entirely on the two things Lightning cannot match — parameter scale and sight — and the case for Lightning rests on four independently measured numbers.

What the numbers cannot tell you is whether any of it has gone wrong in the field: the documented AI incidents and agent failures around models in this class are collected in the OrcaRouter AI Incident Archive, where each entry carries its own sources, severity, confidence grade and disputed flag.