Nemotron 3.5 Bolt 24GB: Apa yang Sebenarnya NVIDIA Masukkan ke dalam File 24GB

The headline number is not the parameter count. It is the byte count. On 15 October 2026 N​VIDIA released N​VIDIA N​emotron 3.5 Bolt 74B-A9B 24GB, a vision-language agent model whose 74 billion parameters occupy 23.75GB in the format most people will actually download — a 22.02GB GGUF plus a 1.73GB vision projector file. Both figures come from the published repository's own file listing, not from the blog post, and they are the reason the name ends in a number. This is what that number buys, what it hides, and what it does not include.

Apa yang dirilis, dan kapan

N​VIDIA's newsroom framed 15 October as a family expansion rather than a single-model launch: N​emotron 3.5 Super VL, a 120-billion-parameter multimodal model at 12 billion active parameters, arrived alongside the Bolt checkpoint, and a much smaller sibling, N​emotron 3.5 Bolt E4B 4GB, is dated for 23 October. Bolt joins N​emotron 3.5 Lightning (August) and N​emotron 3 Ultra in a family that now covers data-centre reasoning, high-throughput serving and local edge agents.

The release is genuinely new, and it is also slightly older than the announcement. The Hugging Face repository nvidia/N​VIDIA-N​emotron-3.5-Bolt-74B-A9B-24GB was created on 7 October 2026; the model card inside it carries a release date of 15 October, the same date N​VIDIA's own research page is stamped "Published". Eight days of files sitting in public view before a single word of explanation is a normal pattern for a vendor shipping under embargo coordination with hardware — RTX Spark laptops are dated 16 October, the day after the announcement — and it is worth knowing which half of this model's public record came from a marketing decision and which came from a file listing.

Bagaimana 74 miliar parameter muat dalam 24 gigabyte

Dua mekanisme, dan keduanya dapat dipisahkan.

The first is structural. Bolt is not a smaller model trained from scratch; it is the Super 3.5 VL parent compressed by an algorithm N​VIDIA calls Star Elastic, a router-guided architecture search that had to satisfy one constraint — a deployment footprint below 24GB at batch size 1 and 128K context — while losing as little knowledge-distillation accuracy as possible. What it produced is a 74B total / 9B active mixture-of-experts hybrid, with the expert top-K fixed at 9 where the parent used 22. N​VIDIA's own framing of the result is blunt and checkable: for a fixed memory budget, compressing a large model beats training a small one.

Yang kedua adalah presisi, dan di sinilah sebagian besar penghematan berada.

• Bobot pakar — lapisan FC1 dan FC2 yang dirutekan dan dibagikan dikuantisasi ke IQ2_XXS pada 2,0625 bit per elemen, yang merupakan sebagian besar model berdasarkan jumlah parameter.

• Lapisan Mamba — NVFP4, format floating-point 4-bit, dan hanya pada lapisan di mana presisi ekstra tersebut sepadan dengan tambahan byte.

• Embedding, kepala keluaran, atensi, dan bobot MoE laten — Q8_0.

• Pemulihan — distilasi sadar kuantisasi alih-alih proses pasca-hoc pembulatan ke nilai terdekat, yang merupakan perbedaan antara model terkompresi dan model yang rusak.

Baca semua itu bersama-sama dan "24GB" tidak lagi menjadi ringkasan pemasaran, melainkan menjadi alokasi anggaran. Expert 2-bit itu murah per byte dan jumlahnya banyak; lapisan-lapisan yang penting untuk stabilitas tetap memakai 4 atau 8 bit.

Angka 24GB, diperiksa terhadap file-file itu

Klaim itu lolos audit, yang tidak selalu benar.

• Bobot GGUF — 22.018.326.528 byte, atau 22,02GB desimal, dalam satu file.

• Proyektor visi — 1.733.692.736 byte, atau 1.73GB, dikirim terpisah sebagai mmproj, yang merupakan cara llama.cpp menerima input gambar dan video.

• Build Safetensors — model yang sama dalam shard siap-vLLM berjumlah total 24,12GB di 90 file, sedikit di atas angka utama.

• Referensi BF16 — 149,8GB yang tersebar di 63 shard, dipublikasikan secara terpisah untuk siapa pun yang ingin melakukan fine-tuning alih-alih menjalankan inferensi.

Two honest caveats sit on top of that. First, weights are not a running model. The vLLM recipe N​VIDIA publishes pins the KV cache to fp8 and sets the served context to 262,144 tokens with max-num-seqs 1 — a single concurrent sequence. That is the difference between fitting in 24GB and running fast in 24GB, and it is the kind of detail the memory-size headline never carries. Second, 24GB is the number the architecture search was given, not a number measured on your machine: driver overhead, the operating system and whatever else is on the laptop are all outside it, which is precisely why the pitch is "run alongside your apps" rather than "run instead of them".

Klaim kecepatan, dan mesin yang digunakan untuk mengukurnya

N​VIDIA designed for at least 80 tokens per second and reports 83.99 on AIPerf SPEED-Bench-Coding. That number needs three pieces of context before it means anything: it was produced with llama.cpp on Windows 11 ARM x64, on a Surface Laptop Ultra with 128GB of memory, with the LiLiCorr speculative drafter attached and seven drafted tokens per step. The drafter is not optional garnish — it is a 1.13B companion checkpoint whose whole purpose is to raise acceptance length (N​VIDIA reports means of 4.68 under greedy decoding and 4.25 with sampling at concurrency 64 in vLLM). Strip the drafter out and you are measuring a different model.

The second speed figure is the more interesting one, because it is a completion time rather than a rate: 600.80 seconds of replay duration on AA-AgentPerf Local, against 453.97 for Qwen3.6 35B-A3B, 480.68 for Gemma 4 26B-A4B and 1,043.28 for Qwen3.8 27B — all from N​VIDIA's own table, all on the same hardware. Bolt wins that row by finishing the same agent trajectories faster than a smaller model, and loses the throughput row to both of the others at 83.99 against 96.63 and 100.56 tokens per second. Fast and quick are not the same measurement, and this is a model that trades the second for the first.

Apa kata pengukuran independen

One number in the public record about Bolt was not produced by N​VIDIA: the Artificial Analysis Intelligence Index scores it at 21, which puts it fourth of 67 models in its class. That panel also reports no speed and no cost for it, because there is no public API to call — and it lists the model's input modality as text and image, not video, which is a narrower reading than the vendor's.

Halaman yang sama mencatat bahwa model tersebut menghasilkan 230M token selama dievaluasi, yang olehnya disebut sangat verbose dibandingkan median 100M. Penalaran yang menghasilkan lebih banyak token pada tugas yang sama tidak otomatis lebih buruk, tetapi ada biayanya pada perangkat lokal di mana setiap token dihasilkan oleh kipas.

Everything else in circulation is vendor-reported. N​VIDIA's own thirteen-benchmark table, in which Bolt leads its memory class on PinchBench and sits second on WebArena and BrowseComp, compares against models chosen partly for fitting in 24GB at 4-bit — a peer group defined by a constraint Bolt was engineered to satisfy. It is not dishonest; it is also not a ranking of agent models in general, and the one cell that goes the other way is instructive: at 16.90 on AA-Omniscience accuracy, Bolt trails Gemma 4 26B-A4B at 22.17 in the same table.

Apa yang tidak ada di dalam kotak

There is no endpoint. No metered rate, no provider serving it, no way to evaluate it on your own traffic without completing the deployment. What there is instead is runtime support that landed at release rather than months later: llama.cpp and the GGUF checkpoint, Ollama, LM Studio, Unsloth for fine-tuning, and vLLM for the quantised checkpoint on DGX Spark and the BF16 checkpoint elsewhere. N​VIDIA's own blog lists those five by name, and the breadth is the real product here — a 24GB file is only useful if the thing that loads it already exists on your machine.

Jangkauan perangkat keras lebih luas daripada yang disiratkan oleh kisah hari peluncuran tentang laptop RTX Spark: PC GeForce RTX, DGX Spark, DGX Station, dan workstation RTX PRO semuanya disebutkan, dan kartu model mencantumkan Ampere, Hopper, dan Blackwell untuk jalur vLLM.

Lisensi ini adalah OpenMDW-1.1, yang cukup permisif untuk membangun di atasnya dan cukup tidak familiar sehingga tinjauan hukum menjadi langkah nyata, bukan sekadar formalitas — patut diketahui sebelum keputusan arsitektur bergantung padanya.

A copy-paste slip in N​VIDIA's own technical blog puts the checkpoint list under the heading "N​emotron 3.5 Super VL 74B-A9B 24GB" rather than Bolt. It is a typo in a fast ship, not a fork in the family, but it is a useful reminder that vendor pages written on announcement day are drafts with a publication date on them.

Untuk tim yang juga memanggil model yang tidak mereka miliki.

The locally-run half of an agent stack and the hosted half are usually the same project, and Bolt only addresses one of them. OrcaRouter does not host any N​emotron checkpoint — that was checked against our own catalogue rather than assumed — so nothing here is a pitch for a model we serve. What we do is the other side of a mixed deployment: one key across 200-plus models at the provider's list price with no markup added, automatic failover when a provider wobbles mid-request, and a routing DSL for describing a fallback chain as configuration instead of application code. If Bolt runs the local half, that is the half that never appears on a bill; the routing question is what happens to the rest.

Apa yang harus ditonton

Tiga hal akan menentukan reputasi model ini, dan tak satupun ada dalam pengumuman tersebut.

The first is independent latency on real hardware. N​VIDIA's 83.99 tokens per second is a controlled benchmark with a speculative drafter attached; someone will eventually publish what it does on a laptop running a browser and an editor at the same time, and that number will matter more than this one. The second is the E4B variant on 23 October, which tests whether the compression recipe scales down or only works at one size. The third is whether anyone outside N​VIDIA post-trains the BF16 checkpoint — because a 24GB deployment artifact with an open licence and a 150GB customization artifact next to it is a deliberate two-track release, and the second track is the one that decides whether this becomes a platform or a checkpoint.

Untuk saat ini, fakta itu berdiri sendiri. Agen multimodal dengan 74 miliar parameter muat di memori sebuah laptop yang harganya lebih murah daripada workstation, berjalan pada sekitar 84 token per detik dengan drafter yang terpasang, mendapat skor 21 pada satu-satunya indeks independen yang telah mengukurnya, dan tidak dapat dipanggil sebagai API oleh siapa pun. Itu adalah kombinasi yang spesifik dan tidak biasa, dan ada gunanya bersikap tepat tentang separuh mana yang Anda butuhkan.

Tidak ada satu pun dari itu yang hipotetis: kegagalan AI dan insiden agen yang terdokumentasi di balik rilis seperti ini dikumpulkan dalam OrcaRouter AI Incident Archive, di mana setiap entri membawa sumbernya sendiri, tingkat keparahan, peringkat keyakinan, dan penanda status diperdebatkan.