Hero title card: NVIDIA Nemotron 3.5 Lightning 30B A3B, Nvidia's open agent workhorse, explained — with stat chips reading 30B total, 3B active, 1M context
Guides & Insights

NVIDIA Nemotron 3.5 Lightning 30B A3B: Nvidia's Open Agent Workhorse, Explained

Author

jinhao.song

Date Published

Latest models · 20View all models →
Benchmarks: Artificial Analysis · updated daily
Back to all posts

Nvidia announced Nemotron 3.5 Lightning on August 11, 2026, and shipped it with something most model launches leave out: a router. NeMo Switchyard, an open-source library that decides which model should handle the next call inside an agent loop, landed the same day as a 30-billion-parameter mixture-of-experts model with just 3 billion active parameters — which is the point. This explainer is about the checkpoint Nvidia released first, NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16, the raw pre-trained base rather than the chat-tuned reference, and about what the family as a whole tells you about where open model serving is heading.

One sourcing note before the numbers: every benchmark, speed claim, and partner figure in this piece is Nvidia's own, and none of it has been independently reproduced yet — the model is hours old. Treat the performance figures as a vendor's card: informative and directional, not yet a scoreboard.

What Nemotron 3.5 Lightning actually is

Nemotron 3.5 Lightning is the newest and smallest member of Nvidia's open Nemotron 3 family. It is a 30B-parameter MoE with only 3B active per token, built on a hybrid Mamba-2 and MoE architecture with select attention layers and multi-token prediction (MTP) heads baked in during a dedicated pre-training stage. Nvidia says it was distilled from its frontier Nemotron 3 Ultra model and trained on more than 20 trillion tokens, and it supports a context window of up to 1 million tokens.

The design goal is latency and cost at scale, not frontier generality. Nvidia positions Lightning for the "execution layer" of always-on agents — the high-volume, lower-difficulty calls an agent makes constantly: tool calls, code review, security-alert triage, billing questions, subagent delegation. That is work you do not want a frontier model on every step; you want a cheap, fast one that is good enough. The speed claims follow from the architecture: up to 4× faster output than similar-sized models and about 30% faster agentic task completion, on Nvidia's measurements. Two draft models, DSpark and DFlash, provide speculative decoding tuned for medium- to high-concurrency serving.

The three checkpoints — and which one you want

The launch is really three downloads, and coverage that treats "Nemotron 3.5 Lightning" as one model misses the distinction that matters most to a developer.

The subject of this piece, NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16, is the raw pre-trained checkpoint — no supervised fine-tuning, no RL, no distillation applied. It is a base model in the strict sense: it does not chat out of the box, and it exists to be the starting point for continued pre-training, custom SFT, RL, or distillation. If you intend to build your own model on top of Nvidia's pretraining, this is the file you want.

The sibling release, NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, is the reference weights that ran the full pipeline — pre-training, MTP continuation, SFT, and RL. It is a general-purpose reasoning and chat model, but Nvidia still frames it as a customization starting point rather than a production serving artifact. For serving, the NVFP4 checkpoint (released alongside, sharing the quantization kernels Nvidia uses for Nemotron 3 Ultra) is the one aimed at latency and throughput.

Practical guidance, in one line per checkpoint:

• Base-BF16 — continued pre-training, domain post-training, distillation from scratch. You take it from here.

• BF16 reference — chat- and reasoning-capable, but intended as the base for your own SFT or RL, not as a hosted endpoint.

• NVFP4 — the serving-optimized checkpoint for deployment on Nvidia GPUs.

Most readers reaching for a production API want the tuned models, not the base. The base matters anyway because Nvidia released it under an open license (OpenMDW-1.1) that covers weights, training data, and recipes — the raw pretraining is the part you cannot get from a closed vendor, and it is the part that makes the whole family yours to own.

Screenshot of the Hugging Face model card for NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16, showing the Text Generation task, Transformers, Safetensors and the OpenMDW-1.1 license

The benchmark ledger, honestly labeled

Here is what Nvidia reports for the base checkpoint on its own evaluations: MMLU 78.59, MMLU-Pro (5-shot) 67.94, GSM8K 91.28, HumanEval 77.44, MBPP 78.59, ARC-Challenge 92.66, HellaSwag 85.55, WinoGrande 79.95, and a long-context RULER score of 69.62 at 1M tokens. The tuned reference moves the needle where you would expect: MMLU-Pro 81.94, GPQA Diamond (no tools) 75.44, SWE-bench Verified 51.56, Terminal-Bench 2.1 24.58.

The headline speed numbers are also vendor-reported: up to 4× faster output and 30% faster agentic task completion than models in its class, plus a PinchBench result of 86% accuracy on a 10,000-task run, roughly 30% faster than Qwen3.6-35B at similar accuracy. Nvidia also cites an Artificial Analysis Intelligence Index of 24, tied with gpt-oss-120b — but note the provenance: that is Nvidia citing a score for a model that, as of August 11, does not yet appear on Artificial Analysis' public leaderboard. Independent labs have not published Lightning numbers.

For a sense of where 24 sits inside the family, the AA-measured indices for Nvidia's other open models are 25.4 for both Nemotron 3 Ultra and Nemotron 3 Super, and 14.2 for Nemotron 3 Nano. Lightning is not a frontier model, and Nvidia is not claiming it is; it is a bet on the speed-and-cost side of the Pareto frontier.

Scoreboard for NVIDIA Nemotron 3.5 Lightning 30B A3B: Active params 3B, Total params 30B, Context up to 1M tokens, MMLU 78.6 (vendor), Output speed up to 4x vs class (vendor), AA Index 24 (Nvidia-cited); footer reads 'Vendor-reported as of Aug 11, 2026 — no independent run yet.'

What it's for: the execution layer of agents

Nvidia's pitch is a division of labor: a frontier model plans and reasons, and a cheap executor handles the thousands of routine calls that make an agent actually run all day. That is why the same announcement includes a routing library — Lightning is meant to be one tier in a larger system, not the whole system.

The partner case studies are Nvidia-supplied, so read them as marketing with numbers attached. The one that best illustrates the model's purpose is CodeRabbit, which says it post-trained Lightning for code review in under three hours for less than $100, gaining about 4% accuracy over a GPT-class baseline at roughly half the inference cost. Whether you trust the exact figures, the shape is the story: a 3B-active model is cheap enough to fine-tune on your own data and serve at scale, which is the whole value proposition.

The cost math

There is no list price for Nemotron 3.5 Lightning, because there is nothing to buy — the weights are free under OpenMDW-1.1, along with training data and recipes. The price is whatever a host charges to serve it, and that number will vary more than the quality does once third-party APIs start carrying the family. For prototyping, Nvidia serves it as a NIM microservice on its own build.nvidia.com platform, which is free to try with a rate limit and no credit card.

Two cost realities matter before you size this up. First, "30B total, 3B active" means inference is cheap per token but the full 30B still has to live in memory on each GPU — serving is not free, and batch concurrency determines how fast it actually runs. Second, when an open-weight model lands on several hosted APIs, per-token prices across providers can differ by multiples, and that difference is not a quality signal. That is where a pass-through gateway earns its keep: OrcaRouter passes provider list prices straight through at 0% markup, so a price cut at the provider is live the same day and price variance stays on the market's side. Nemotron 3.5 Lightning is not on OrcaRouter yet — the weights landed hours ago — but the pattern for adopting a day-old model safely is to route it behind automatic failover, so a cold-start problem on one host cannot take down a production path.

The router that shipped with it

NeMo Switchyard is worth understanding because it is Nvidia's bet on how open agentic AI should run: not one big model, but a system that sends each step to the most capable and efficient model available. It is an open-source library you integrate into your harness, with knobs to tune toward quality, latency, or cost. Nvidia's own benchmark claims frontier-level accuracy at close to one-third the cost of a frontier model running alone.

The partner results, again Nvidia-reported, are directionally consistent: Boomi says it routed 59% of traffic to a 5× faster specialized model with 21% lower later-turn latency; Cognition reports 28% lower average cost; Ramp says it matched frontier-model performance on its own SWE-bench at 58% lower cost. The most honest data point is LangChain: 74% lower cost across 145 multi-step agent tasks with only 7% of calls going to a frontier model — at a 6% accuracy tradeoff. Routing is not free; it is a deliberate trade.

Switchyard is a library, which means it is the build-it-yourself path: you define the policy, integrate it, operate it. The managed alternative is a hosted routing API, where one key covers a catalogue of 200-plus models, failover is a default rather than something you wire up, and a routing DSL lets you compose several models into a single call. Nvidia's own test numbers — 59%, 74%, 58% — are the strongest argument for that pattern, whatever layer you run it on.

How to run it today

Day-one support is broad. The checkpoints run locally on RTX PCs, DGX Spark, DGX Station, and Jetson hardware, and in the data center on Hopper and Blackwell GPUs. The announced ecosystem covers vLLM, SGLang, llama.cpp, Ollama, LM Studio, Unsloth, and TensorRT-LLM, plus a NIM container for Nvidia infrastructure. For post-training, Nvidia also released an open RL dataset, Nemotron-RL-Agentic-Terminal-Pivot, aimed at coding-agent capabilities — useful if you plan to do your own RL step on top of the base.

Screenshot of the NVIDIA developer blog post announcing Nemotron 3.5 Lightning for long-running agent workloads, dated August 11, 2026

Who should act now

Three groups get something from this release, and they should do different things. If you build models — continued pre-training, domain post-training, distillation — the Base-BF16 checkpoint is the interesting download, and it is genuinely new: Nvidia has not opened a raw 30B-class pretraining under this license before. If you serve models, the NVFP4 checkpoint and the NIM path are worth benchmarking against what you already run for agent workloads; the 3B-active profile is the entire pitch. And if you are an application developer, the honest answer is to wait a beat: the tuned models are not yet on hosted APIs in any stable form, independent numbers are weeks away, and the cost picture will be clearer once the market prices an open 30B MoE.

What to watch next: the first independent run on Artificial Analysis, which will confirm or deflate the Index-24 claim; the prices the first hosting providers set for the tuned checkpoints; and how often Nvidia's own Switchyard numbers show up in other people's benchmarks. This is a launch about efficiency and routing, not about a new frontier — and that, not the spec sheet, is the story.

FAQ

Can I use the Base-BF16 checkpoint for chat out of the box?

No. NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16 is the raw pre-trained checkpoint with no SFT or RL applied — it is not instruction-tuned. The chat- and reasoning-capable option is the sibling NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 reference release, which ran the full four-stage pipeline; the base exists to be post-trained further, not deployed as-is.

Is the Artificial Analysis Index of 24 an independent score?

No — not yet. That figure is Nvidia citing a score for a model that, as of August 11, 2026, does not yet appear on Artificial Analysis' public leaderboard. Independent labs have not published Nemotron 3.5 Lightning results, so treat every number in the launch materials as vendor-reported until a third-party run appears.