Nemotron 3.5 Bolt E4B 4GB: What Actually Shipped, and Why the Name Misleads

NVIDIA-Nemotron-3.5-Bolt-E4B-4GB went live on Hugging Face on 23 October 2026, and the first thing worth saying about it is arithmetic: the file is 6.18GB. The name says 4GB. Both numbers are real, they measure different things, and the gap between them is the most useful fact in the release.

NVIDIA's technical blog calls 4GB the intended memory envelope for the model's target workloads — a budget for what the model should occupy while an application runs beside it. It is not a parameter count and not a download size. The checkpoint itself is 8B total with 4B effective parameters, post-trained from Gemma-4-E4B-it, and published in exactly two precisions: a 15.88GB BF16 build and a 6.18GB Q4_K_M GGUF. If you arrived looking for a four-gigabyte file to drop on a device, there isn't one in the repository.

That is worth stating plainly because the rest of the release is unusually legible. The model card is detailed, the technical report is public, and the measured latency table covers four devices. What is missing is independent verification, and the card says so itself.

The release, reconstructed from what is public

This one did not get a launch. It got a footnote and then a blog post.

• 5 October 2026 — the Hugging Face repositories for the Q4_K_M and BF16 builds are created. Neither is populated with weights.

• 15 October 2026 — NVIDIA announces the Nemotron 3.5 family: Super VL 120B-A12B and Bolt 74B-A9B 24GB, both available that day. The E4B gets two sentences and a date: coming 23 October.

• 23 October 2026 — the Q4_K_M and BF16 weights are published, the model card goes up, and NVIDIA's research lab posts a technical blog and a technical report. There is no corporate press release, and the launch coverage that followed the 15 October announcement does not return for this one.

So "shipped quietly" is accurate, but "secret" would not be. NVIDIA named the model, dated it, and then documented it properly on the day. What it did not do was put the E4B in front of the same reporters who covered its siblings. The practical consequence for anyone reading this in the first weeks is that almost every claim below traces back to one source: NVIDIA.

Four gigabytes is a budget, not a download

The published file sizes are unambiguous once you look at the repository rather than the name.

• Q4_K_M GGUF — 6,180,446,208 bytes, or 6.18GB in decimal terms, in a single file named Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf.

• BF16 safetensors — 15,882,494,698 bytes, or 15.88GB, in one shard, plus a 110MB frozen shared-KV file.

• No 4GB artifact — there is no smaller quantization in the repository, and the card does not publish a file that hits the number in the name.

What NVIDIA does publish is a batch-1 llama.cpp measurement on four devices, and that table is where the 4GB claim becomes concrete:

• RTX 5070 12GB — 3.5 to 4.3GB of VRAM, 548 to 1,238ms time to first token, 106.2 to 133.4 tokens per second.

• RTX Spark 32GB — 3.1 to 4.2GB of VRAM, 930 to 1,988ms to first token, 52.0 to 67.0 tokens per second.

• Jetson Orin Nano 8GB — 4.2 to 4.9GB of VRAM, 5,785 to 6,259ms to first token, 13.0 to 15.5 tokens per second.

• Jetson AGX Orin 32GB — 4.2 to 5.0GB of VRAM, 3,687 to 4,091ms to first token, 20.3 to 24.0 tokens per second.

The VRAM figures sit below the 6.18GB file size, which means the runtime is not holding the entire file resident on the GPU in those runs. The card does not explain the mechanism, and this is a good place to resist inventing one: memory-mapped weights, partial offload, and KV-cache sizing all interact, and NVIDIA has not published the split. What can be said is that the 4GB envelope is a measured result on specific hardware under a specific runtime, not a property of the download.

The Orin Nano row is the one to read twice if the pitch is voice assistance. Thirteen to fifteen tokens per second with nearly six seconds to first token is a workable embedded agent and a poor real-time conversation partner. The RTX 5070 row is the opposite: fast enough for interactive use, on a desktop GPU.

What NVIDIA actually changed in Gemma 4 E4B

The architecture is inherited wholesale. This is Gemma-4-E4B, an 8B-total mixture with 4B effective parameters, and NVIDIA's contribution is post-training rather than a new backbone. The card lists the work as reinforcement learning from verifiable rewards and human feedback through NeMo-RL and NeMo-Gym, across tool calling and schemas, instruction following and hierarchy, structured outputs, free-form formatting, and generative reward modelling. Training batches held a 2:1 ratio of non-reasoning to reasoning samples, and NVIDIA increased length penalties to control verbosity — a detail that shows up later in the benchmark table.

The Q4_K_M build was produced with llama.cpp using imatrix calibration on roughly one million tokens of RL rollouts. There is also a DFlash drafter for speculative decoding, which NVIDIA says delivered up to 21% more throughput than Gemma-4-E4B's original multi-token-prediction drafter. The drafter is a separate repository, and at the time of checking it returned HTTP 401 to anonymous requests, so treat it as an additional access step rather than a same-day download.

Two chat-template switches do most of the behavioural work. enable_thinking turns reasoning on or off and accepts a runtime thinking budget; conversational_mode adds a spoken-conversation system instruction for short, low-formatting replies. That second switch is the tell. This model is built to talk, not to fill a context window with analysis.

Context is 64K in and 64K out. Language coverage is 34 in the Hugging Face metadata and 35 in the technical blog — a small inconsistency, not a substantive one, and the post-training card names English, French, German, Italian, Japanese, Spanish, and Chinese as primary.

The licence is a stack rather than a single document: OpenMDW-1.1 for NVIDIA's contribution, Apache 2.0 for the Gemma 4 component. The card states the model is ready for commercial or non-commercial use. OpenMDW is permissive but not OSI-approved, which matters to legal teams more than to developers.

The benchmark table, read against the models it names

NVIDIA evaluated the model against Gemma4 E4B-it, Qwen3.5 4B, and its own Nemotron 3 Nano-4B, under a consistent NeMo-Gym harness. Every figure below is vendor-reported and unreproduced. The card says the numbers "may differ from vendors' self-reported numbers," which is an honest caveat and also a reminder that the comparison set is NVIDIA's choice.

The wins are concentrated in conversation and control:

• RoleBench Custom — 67.05 for Bolt E4B against 53.66 for Qwen3.5 4B, 49.95 for Gemma4 E4B-it, and 12.02 for Nano-4B. This is the persona benchmark, and the margin is the widest in the table.

• IHEval, instruction hierarchy — 84.79 against 60.57, 67.04, and 51.31. A 17-point lead over the nearest competitor is unusual at this size.

• When2Call Multi-Turn — 63.36 against 37.85 for Qwen3.5 4B, 62.90 for Gemma4, and 27.70 for Nano-4B. Knowing when not to call a tool is a conversational skill as much as an agentic one.

• RelyToolBench — 63.16 against 43.35, 61.60, and 41.20. Grounding in tool output, again a conversation-adjacent skill.

• MultiChallenge — 53.10 against 51.76, 42.08, and 35.30. A narrow win on multi-turn challenge handling.

The losses are equally clear, and they are the ones that matter if you want an agent rather than a character:

• Tau2-Bench — 55.10 against 75.06 for Qwen3.5 4B. This is a 20-point deficit on multi-turn agentic task completion, and no amount of persona quality closes it.

• RULER at 64K — 86.99 against 95.15 for Qwen3.5 4B and 91.51 for Nano-4B. The model accepts 64K of context; it does not retrieve across it as reliably as its peers.

• LongBench v2 at 48K — 50.50 against 59.93 for Qwen3.5 4B. Same story on long-document reasoning.

• BFCL-v3 — 68.51 against 70.95 for Qwen3.5 4B. A small deficit on function calling, and a win over Gemma4's 66.39.

• EQ-Bench v3 — 39.40 against 41.17 for Gemma4 E4B-it. The one emotional-intelligence row where the parent beats the child.

The Q4_K_M column is more interesting than a quantization table usually is. Compression did not uniformly cost accuracy: MultiChallenge rises from 53.10 to 55.60, RoleBench from 67.05 to 71.21, ToolSandbox from 68.77 to 74.13, and RelyToolBench from 63.16 to 69.29. It falls on Tau2 from 55.10 to 52.32, on LongMemEval from 72.10 to 69.92, and on RULER from 86.99 to 85.76. A four-bit build that improves several conversational scores is plausible when the calibration data comes from the model's own RL rollouts, but it is still a vendor measurement, and the benchmark-level noise in a 4B model is large enough that these deltas should not be read as a precision ladder.

Two rows deserve their own flag. Brevity Bench and When2Call Multi-Turn are evaluations NVIDIA describes as addressing gaps in existing benchmarks, and the blog says they will be released at a later date. They are not independently reproducible today — which matters most for Brevity, because that is the row carrying the fewer-output-tokens story.

NVIDIA's headline claim is that the model averages about 40% better than Nemotron 3 Nano-4B and roughly 10% better than peer models, with up to 80% fewer output tokens in reasoning-enabled comparisons. The 80% figure is a verbosity claim, not an accuracy claim, and it lines up with the Brevity Benchmark score of 82.52 — which NVIDIA built into its own suite. Take the direction seriously and the exact percentage lightly.

What the repository will not tell you

The card is candid about its own gaps, and the gaps are worth listing because they define how much confidence the release supports.

• No independent evaluation. Artificial Analysis has no model page for the E4B at the time of writing, and no third party has published a reproduction of the 21-row benchmark table.

• No hosted price. The card dates a Build.NVIDIA.com listing to 23 October, but no per-token price is published there or in the card. If you want an API, you are negotiating or using a free tier, not reading a rate card.

• Deployment is marked "llama.cpp (recipe pending)" even though the quick start contains working llama.cpp commands. That is a documentation contradiction, not a technical one, but it is the kind of thing that trips up an automated deployment pipeline.

• The 4GB target is not demonstrated as a file. The measured VRAM envelope is the evidence; the download is 6.18GB.

• Safety, bias, explainability, and privacy subcards exist as separate files. This article does not audit them, and neither does any independent body yet.

None of that makes the release untrustworthy. It makes it early. A model card written by the team that trained the model is the best available description of what the model is for; it is not evidence that the model does it.

Where a local 4B model stops, and what to put behind it

The honest architecture for this class of model is a split one. A 4B conversational checkpoint on a Jetson or an RTX card is excellent at the reflex layer: wake-word responses, persona dialogue, short tool calls, input grounding, the parts of an interface that cannot afford a network round trip. It is not the thing you want answering a question that needs 200K of context or a ten-step agentic plan, and its own benchmark table says so.

That leaves every deployment with a second question: what handles the requests the local model should not. Putting a hosted model behind the same application through OrcaRouter means one API for 200+ models, 0% markup on provider list price, and automatic failover when a provider degrades — so the escalation path is a routing decision rather than a second SDK, a second key, and a second billing relationship. The local model stays the fast path; the hosted model is the fallback for the cases the E4B was never trained for.

What would change this picture

Three things, in order of usefulness. An independent benchmark run on the same harness would settle whether the conversation-heavy profile survives outside NVIDIA's evaluation. A published 4GB-or-smaller artifact would make the name literal. And a price on the Build endpoint would turn the model from a download into a service you can compare.

Until then, the correct reading is narrow and useful: NVIDIA post-trained Gemma 4 E4B into a demonstrably better conversational small model, published the evidence and the gaps in the same document, and shipped it a week after the family launch with almost no noise. The 4GB in the name is a target that the measured runtime meets and the download does not. Plan your storage around 6.18GB and your expectations around 13 to 133 tokens per second, depending on which device you actually own.