Nemotron 3.5 Transcribe vs Nemotron 3.5 Transcribe 30B: Two Names, One Checkpoint, Four Files

If a catalogue, a comparison site or your own procurement sheet lists Nemotron 3.5 Transcribe and Nemotron 3.5 Transcribe 30B as two entries, it is listing one model twice. The published repository is NVIDIA-Nemotron-3.5-Transcribe-30B-A4B, NVIDIA's card gives the total as 34B parameters and the active count as about 4.3B, and the plain name in every headline is the same artefact with the size dropped. There is no separate 30B model to compare against, which makes this less a head-to-head than a naming problem with a deployment decision buried inside it.

That decision is real, though, and it is the reason the two names keep turning up side by side. One name is what you search for and what a model catalogue or api listing might display. The other is what you download, in one of four precisions, at one of four memory footprints, and the choice between those files is the only comparison this model actually asks you to make. What follows is the reconciliation: what the 30B counts, why the card says 34B, and which file fits the GPU you have rather than the one the launch page assumes.

The name that disagrees with itself

Two numbers are in circulation and both are official. The repository name ends in 30B-A4B, following the convention NVIDIA uses for its mixture-of-experts line: total parameters first, active parameters second. The model card, on the same page, states 34B total and roughly 4.3B active, explicitly including the audio encoder. The research page repeats the 30B-A4B name in its heading and the 4.3B active figure in its body.

The card also names the pieces, which is where the arithmetic starts to matter. The language backbone is Nemotron 3.5 Lightning 30B-A3B, a Mamba-2 and Transformer hybrid MoE with 3.5B active parameters. The joint audio encoder — a Transformer ASR encoder plus a speaker encoder — is about 800M. Add those and you get roughly 30.8B, not 34B. Either the 34B counts something NVIDIA has not itemised, or the 30B in the repository name is a rounded label for a backbone that is larger in fact than in its name. NVIDIA has not published the reconciliation, and no third-party parameter audit of the full checkpoint exists yet.

What that means for a buyer is narrower than it sounds. Whether the system is 31B or 34B does not change which card it fits on, because the file size is governed by precision far more than by the last three billion parameters. It does mean that any spec sheet quoting "30B" and any quoting "34B" are describing the same download, and a procurement comparison built on the difference is comparing a rounding convention.

What the 30B suffix is actually counting

The suffix counts a language model, not a transcriber. Nemotron 3.5 Transcribe is a speech-augmented LLM: the audio goes through a Transformer encoder, a frozen Nemotron 3 Diarization model contributes speaker representations through what NVIDIA calls a parallel expert encoder, and the hybrid MoE decoder writes the transcript. The 30B label belongs to the decoder and its embedding stack. The speech path — encoder, diarizer, CTC timestamp head — rides on top and is not what the name advertises.

That is why the model behaves like an LLM in ways a classical ASR system does not. It takes prompts. It answers questions about the audio. It translates across 31 pairs to and from English inside the same pass. It holds a 65,536-token context so that an hour of speech fits without chunking. None of that comes from a 30B decoder being large; it comes from the decoder being a general language model at all, with speech projected into its input space.

It also explains the absence everyone keeps looking for. There is no 30B-only sibling, no smaller dense Transcribe, and no larger one. The repository carries precisions of a single model, not a family. The only other sizes in circulation are quantisations produced by the community, and one genuinely smaller streaming recogniser — Nemotron Speech Streaming EN 0.6B — that is a different model with English-only coverage and no diarisation, not a mini version of this one.

The four files behind the name, and what each one costs in memory

This is the comparison the two names are actually pointing at. NVIDIA publishes three precisions and the community has added a fourth delivery format; the memory figures below are from the published checkpoints and the community repositories, not from a single vendor table.

• BF16 — the reference checkpoint, roughly 67-68GB on disk, served on a single H100 80GB. This is the version the published benchmark numbers were produced on, and the one to use when you need to reproduce them.

• FP8 — about 35GB, which moves the floor to a 48GB card such as an L40S. Accuracy loss against BF16 is small in NVIDIA's own framing, but it is NVIDIA's framing; no independent FP8 evaluation has been published.

• NVFP4 — about 21GB, small enough for an RTX 5090 at 32GB or an RTX PRO 6000. This is the precision behind NVIDIA's headline throughput claim of 2,238 seconds of audio per second on a single RTX PRO 6000 with the DFlash2 draft model attached.

• GGUF — four community decoders between 19.7GB and 35GB plus a mandatory 1.67GB F16 encoder file. The 19.7GB IQ4_NL build is the one aimed at 24GB cards, CPU inference, Metal and Vulkan; the 19.7GB NVFP4 build is aimed at Blackwell; Q5 sits near 29GB and Q8 near 35GB. An optional 1.15GB DFlash2 drafter adds speculative decoding on the CUDA path.

• MLX — community 4-bit to 8-bit ports between roughly 19GB and 35GB for Apple silicon, with one author reporting about 154 tokens per second on the 4-bit build. Useful for evaluation on a Mac, not a production serving path.

What runs them, and the one runtime mistake people make

The GGUF files are the trap. A file ending in .gguf does not imply llama.cpp, Ollama or LM Studio, and none of those three run this model: it needs a speech-LLM runtime with the audio encoder wired in. The working paths at release are narrower than the format list suggests.

• vLLM 0.32.0 — the primary path and the only one that exposes the full output surface, including word timestamps through the V1 runner and timestamp middleware. Speaker-attributed output and translation live here.

• SGLang-Omni 0.2.0 on SGLang 0.5.22 — whole-file transcription through an OpenAI-compatible audio endpoint, without word timestamps, speaker attribution, translation or speculative decoding.

• NeMo Speech — the fine-tuning toolkit, with NVIDIA quoting 0.5 to 2.3 GPU-hours per task on a single GPU. The published LoRA recipes run on one card.

• NeMo-Speech.cpp and speech.cpp — the C++ path the GGUF builders target, and the only one that loads those quants. Both the 19.7GB and 35GB files above are distributed for it, and the repositories' READMEs are the honest source for what fits where.

• Transformers — in-process use from a source install, adequate for evaluation, not for throughput.

The mistake is downloading the smallest quant, pointing llama.cpp at it, and concluding the model is broken. It is not broken; it is a speech LLM with an encoder that the text runtimes have no concept of. If the runtime does not list Nemotron 3.5 Transcribe in its supported architectures, the file will not load regardless of how small it is.

Where each file fits, in plain terms

Work backwards from the hardware rather than forwards from the model card. The card's stated baseline — one H100 80GB at BF16 — is the maximum-fidelity path, not the minimum one.

• 24GB card, RTX 4090 or L4 class — the 19.7GB IQ4_NL GGUF with its 1.67GB encoder. Leave headroom for the KV cache and the audio buffer; 24GB is tight but documented as working.

• 32GB card, RTX 5090 or RTX PRO 6000 — the 21GB NVFP4 checkpoint, which is the configuration NVIDIA's throughput claim describes, with the DFlash2 drafter if you want speculative decoding.

• 48GB card, L40S or A6000 — the 35GB FP8 checkpoint, the least awkward compromise between fidelity and cost.

• 80GB accelerator, H100 or A100 — BF16, the reference configuration, and the only one that reproduces the published WER table without a precision caveat.

• No GPU at all — none of these files. The encoder runs on CPU in the C++ path, but an hour of audio on a 30B-A3B decoder without acceleration is a batch job measured in hours, not a service.

One constraint survives every precision: the input is 16kHz mono PCM and the ceiling is one hour of audio per request. Downsampling and channel selection happen before the model sees anything, and a stereo boardroom recording gets mixed to mono unless you split the channels yourself. That is a preprocessing decision, not a checkpoint decision, and getting it wrong costs more transcription quality than the BF16-to-FP8 step does.

The checks to run when a listing shows both names

Since the duplicate-name situation is real and will recur in catalogues and comparison pages, a short verification routine is worth more than another paragraph of explanation.

• Repository id — the only official artefact is NVIDIA-Nemotron-3.5-Transcribe-30B-A4B. A row named Nemotron 3.5 Transcribe 30B with a different repo id is either a quantisation or a different model entirely.

• Licence string — OpenMDW-1.1 on the official release, commercial use permitted. A listing that says research-only or non-commercial is describing something else.

• Context and audio window — 65,536 tokens and one hour. A listing quoting a longer window is wrong or is describing a different checkpoint.

• Parameter claim — 30B-A4B in the name, 34B in the card, 4.3B active in both. A listing quoting 4B total has confused the active count for the total, which is the most common transcription of this model on the web.

• File list — BF16, FP8 and NVFP4 from NVIDIA, plus a DFlash2 draft model. If a page offers "Nemotron 3.5 Transcribe 30B" as a single downloadable file and no precisions, it is a community quant wearing the vendor's name.

The routing question that is actually separate

Once the file is chosen and the runtime is working, the deployment question is over and an older one returns: what happens to the text. A transcript is raw material for summarisation, entity extraction, redaction, search indexing and question answering, and those calls are ordinary text-model traffic with ordinary text-model problems — rate limits, upstream outages, a cheap classification model that would do for nine tenths of the volume.

OrcaRouter does not serve Nemotron 3.5 Transcribe, and its catalogue does not carry an entry under either name; if you see two rows for this model anywhere, the repository check above is the fastest way to settle it. What the platform does provide is the layer after the transcript exists: one OpenAI-compatible key across 200+ text and vision models at provider list price with 0% markup, automatic failover across providers, and a routing DSL that composes several models into one call so the cheap extraction pass and the expensive judgement pass can live behind a single endpoint. The GPU transcribes; routing decides what the transcript is worth.

The suffix is a file-size label, not a second model

The cleanest way to hold both names in your head is this: Nemotron 3.5 Transcribe is the product, Nemotron 3.5 Transcribe 30B is the repository, and the 30B counts a decoder while the card counts a system. The gap between 30.8B and 34B is an unresolved detail NVIDIA should reconcile, but it changes no decision a buyer makes, because the decision is made in gigabytes and in supported runtimes, not in the third digit of a parameter count.

Pick the precision your card can hold, confirm the runtime lists the architecture, and treat any listing that presents the two names as separate models as a catalogue error rather than a choice. The real fork in this release — the one that would justify a genuine head-to-head — is the 0.6B streaming recogniser on one side and this 30B offline transcriber on the other. That comparison is about latency and coverage, and it is worth making. Comparing a model to its own filename is not.