Nemotron 3.5 Bolt 24GB：NVIDIAが24GBのファイルに実際に入れたもの

The headline number is not the parameter count. It is the byte count. On 15 October 2026 N​VIDIA released N​VIDIA N​emotron 3.5 Bolt 74B-A9B 24GB, a vision-language agent model whose 74 billion parameters occupy 23.75GB in the format most people will actually download — a 22.02GB GGUF plus a 1.73GB vision projector file. Both figures come from the published repository's own file listing, not from the blog post, and they are the reason the name ends in a number. This is what that number buys, what it hides, and what it does not include.

何がリリースされたのか、そしていつか

N​VIDIA's newsroom framed 15 October as a family expansion rather than a single-model launch: N​emotron 3.5 Super VL, a 120-billion-parameter multimodal model at 12 billion active parameters, arrived alongside the Bolt checkpoint, and a much smaller sibling, N​emotron 3.5 Bolt E4B 4GB, is dated for 23 October. Bolt joins N​emotron 3.5 Lightning (August) and N​emotron 3 Ultra in a family that now covers data-centre reasoning, high-throughput serving and local edge agents.

The release is genuinely new, and it is also slightly older than the announcement. The Hugging Face repository nvidia/N​VIDIA-N​emotron-3.5-Bolt-74B-A9B-24GB was created on 7 October 2026; the model card inside it carries a release date of 15 October, the same date N​VIDIA's own research page is stamped "Published". Eight days of files sitting in public view before a single word of explanation is a normal pattern for a vendor shipping under embargo coordination with hardware — RTX Spark laptops are dated 16 October, the day after the announcement — and it is worth knowing which half of this model's public record came from a marketing decision and which came from a file listing.

740億個のパラメータがどのようにして24ギガバイトに収まるのか

2つのメカニズムであり、それらは分離可能である。

The first is structural. Bolt is not a smaller model trained from scratch; it is the Super 3.5 VL parent compressed by an algorithm N​VIDIA calls Star Elastic, a router-guided architecture search that had to satisfy one constraint — a deployment footprint below 24GB at batch size 1 and 128K context — while losing as little knowledge-distillation accuracy as possible. What it produced is a 74B total / 9B active mixture-of-experts hybrid, with the expert top-K fixed at 9 where the parent used 22. N​VIDIA's own framing of the result is blunt and checkable: for a fixed memory budget, compressing a large model beats training a small one.

二つ目は精度であり、節約の大半がここで生まれます。

• エキスパート重み — ルーティングされた FC1 および FC2 層と共有された FC1 および FC2 層は、要素あたり 2.0625 ビットで IQ2_XXS に量子化されており、パラメータ数で見るとモデルの大部分を占めます。

• Mambaレイヤー — NVFP4、4ビット浮動小数点フォーマット、しかも追加の精度がより多くのバイトに見合うレイヤーにのみ。

• 埋め込み、出力ヘッド、アテンション、および潜在MoE重み — Q8_0。

• リカバリー — 事後的な最近接丸め処理ではなく、量子化を意識した蒸留。これが、圧縮されたモデルと壊れたモデルの違いを生む。

これらをまとめて読むと、「24GB」はマーケティング上の要約ではなくなり、予算の配分へと変わる。2ビットのエキスパートは1バイトあたり安価で数も多く、安定性にとって重要な層は4ビットか8ビットを維持する。

ファイルと照合した24GBという数値

その主張は監査を通過するが、それは必ずしも当てはまらない。

• GGUFウェイト — 22,018,326,528バイト（10進数で22.02GB）、単一ファイル内。

• ビジョンプロジェクター — 1,733,692,736バイト、つまり1.73GBで、mmprojとして別途配布されています。これがllama.cppで画像や動画を入力する仕組みです。

• Safetensorsビルド — 同じモデルをvLLM対応のシャードに分割すると、90ファイル合計で24.12GBとなり、公称値をわずかに上回ります。

• BF16リファレンス — 63のシャードにわたる149.8GB、推論ではなくファインチューニングを希望する人向けに別途公開。

Two honest caveats sit on top of that. First, weights are not a running model. The vLLM recipe N​VIDIA publishes pins the KV cache to fp8 and sets the served context to 262,144 tokens with max-num-seqs 1 — a single concurrent sequence. That is the difference between fitting in 24GB and running fast in 24GB, and it is the kind of detail the memory-size headline never carries. Second, 24GB is the number the architecture search was given, not a number measured on your machine: driver overhead, the operating system and whatever else is on the laptop are all outside it, which is precisely why the pitch is "run alongside your apps" rather than "run instead of them".

速度の主張と、それを測定したマシン

N​VIDIA designed for at least 80 tokens per second and reports 83.99 on AIPerf SPEED-Bench-Coding. That number needs three pieces of context before it means anything: it was produced with llama.cpp on Windows 11 ARM x64, on a Surface Laptop Ultra with 128GB of memory, with the LiLiCorr speculative drafter attached and seven drafted tokens per step. The drafter is not optional garnish — it is a 1.13B companion checkpoint whose whole purpose is to raise acceptance length (N​VIDIA reports means of 4.68 under greedy decoding and 4.25 with sampling at concurrency 64 in vLLM). Strip the drafter out and you are measuring a different model.

The second speed figure is the more interesting one, because it is a completion time rather than a rate: 600.80 seconds of replay duration on AA-AgentPerf Local, against 453.97 for Qwen3.6 35B-A3B, 480.68 for Gemma 4 26B-A4B and 1,043.28 for Qwen3.8 27B — all from N​VIDIA's own table, all on the same hardware. Bolt wins that row by finishing the same agent trajectories faster than a smaller model, and loses the throughput row to both of the others at 83.99 against 96.63 and 100.56 tokens per second. Fast and quick are not the same measurement, and this is a model that trades the second for the first.

独立した測定が示すもの

One number in the public record about Bolt was not produced by N​VIDIA: the Artificial Analysis Intelligence Index scores it at 21, which puts it fourth of 67 models in its class. That panel also reports no speed and no cost for it, because there is no public API to call — and it lists the model's input modality as text and image, not video, which is a narrower reading than the vendor's.

同じページは、モデルが評価中に2億3000万トークンを生成したと指摘しており、それを中央値の1億トークンに対して非常に冗長だと評している。同じタスクでより多くのトークンを生成する推論が自動的に劣っているわけではないが、すべてのトークンがファンによって生成されるローカルデバイスでは、それには何らかのコストがかかる。

Everything else in circulation is vendor-reported. N​VIDIA's own thirteen-benchmark table, in which Bolt leads its memory class on PinchBench and sits second on WebArena and BrowseComp, compares against models chosen partly for fitting in 24GB at 4-bit — a peer group defined by a constraint Bolt was engineered to satisfy. It is not dishonest; it is also not a ranking of agent models in general, and the one cell that goes the other way is instructive: at 16.90 on AA-Omniscience accuracy, Bolt trails Gemma 4 26B-A4B at 22.17 in the same table.

箱の中に何が入っていませんか

There is no endpoint. No metered rate, no provider serving it, no way to evaluate it on your own traffic without completing the deployment. What there is instead is runtime support that landed at release rather than months later: llama.cpp and the GGUF checkpoint, Ollama, LM Studio, Unsloth for fine-tuning, and vLLM for the quantised checkpoint on DGX Spark and the BF16 checkpoint elsewhere. N​VIDIA's own blog lists those five by name, and the breadth is the real product here — a 24GB file is only useful if the thing that loads it already exists on your machine.

ハードウェアの対応範囲は、RTX Spark ノートPCに関する発売当初の話が示唆するよりも広い。GeForce RTX PC、DGX Spark、DGX Station、RTX PRO ワークステーションがすべて挙げられており、モデルカードには vLLM パス用として Ampere、Hopper、Blackwell が記載されている。

ライセンスは OpenMDW-1.1 で、その上に構築できるほど寛容でありながら、法務レビューが形式上の手続きではなく実際の工程となるほど馴染みが薄い — アーキテクチャ上の決定がそれに依存する前に知っておく価値がある。

A copy-paste slip in N​VIDIA's own technical blog puts the checkpoint list under the heading "N​emotron 3.5 Super VL 74B-A9B 24GB" rather than Bolt. It is a typo in a fast ship, not a fork in the family, but it is a useful reminder that vendor pages written on announcement day are drafts with a publication date on them.

自分たちが所有していないモデルも呼び出すチーム向け

The locally-run half of an agent stack and the hosted half are usually the same project, and Bolt only addresses one of them. OrcaRouter does not host any N​emotron checkpoint — that was checked against our own catalogue rather than assumed — so nothing here is a pitch for a model we serve. What we do is the other side of a mixed deployment: one key across 200-plus models at the provider's list price with no markup added, automatic failover when a provider wobbles mid-request, and a routing DSL for describing a fallback chain as configuration instead of application code. If Bolt runs the local half, that is the half that never appears on a bill; the routing question is what happens to the rest.

何を見るべきか

このモデルの評判を決めるのは3つの点であり、そのどれも発表内容には含まれていない。

The first is independent latency on real hardware. N​VIDIA's 83.99 tokens per second is a controlled benchmark with a speculative drafter attached; someone will eventually publish what it does on a laptop running a browser and an editor at the same time, and that number will matter more than this one. The second is the E4B variant on 23 October, which tests whether the compression recipe scales down or only works at one size. The third is whether anyone outside N​VIDIA post-trains the BF16 checkpoint — because a 24GB deployment artifact with an open licence and a 150GB customization artifact next to it is a deliberate two-track release, and the second track is the one that decides whether this becomes a platform or a checkpoint.

現時点では、この事実はそれ単独で成立している。740億パラメータのマルチモーダルエージェントが、ワークステーションより安価なラップトップのメモリに収まり、ドラフターを付ければ毎秒約84トークンで動作し、それを測定した唯一の独立系インデックスで21点を取り、しかも誰もAPIとして呼び出すことができない。これは具体的かつ異例な組み合わせであり、そのどちらの半分が必要なのかを正確に把握する価値がある。

そうした話は決して仮定ではない。このようなリリースの背景にある、記録に残されたAIの障害やエージェントのインシデントはOrcaRouter AIインシデントアーカイブに集められており、各項目には独自の情報源、深刻度、信頼度グレード、係争中フラグが付されています。