Nemotron 3.5 Super vs DeepSeek V4 Pro: the benchmark table NVIDIA did not publish

Open NVIDIA's model card for NVIDIA Nemotron 3.5 Super VL — the 120-billion-parameter open checkpoint the vendor also lists as NVIDIA Nemotron 3.5 Super and that Ollama ships as nemotron-3.5-super — and scroll to the evaluation table. You will find roughly forty benchmarks and seven baseline columns. One of those columns is DeepSeek: it is labelled DeepSeek-V4-Flash0731. DeepSeek's own cheap tier. DeepSeek V4 Pro, the 1.6-trillion-parameter flagship that superseded the April preview in the 0813 revision, does not appear anywhere in the table.

That absence is the most useful fact in this comparison, and it is not an oversight. V4 Pro activates 49 billion parameters per token. Putting it in the same table as a 12-billion-active model would produce a set of numbers that flatter nobody and inform nobody, because the two models are not the same product. What the vendor did instead is the honest version of the exercise: benchmark against the tier you actually intend to displace.

So this page does two things. It reads the comparison NVIDIA published — Nemotron 3.5 Super against DeepSeek's flash tier — and then it is explicit, in the places where the two models really do compete, about what can and cannot be concluded about the flagship.

The two artefacts, side by side

• Total and active parameters — Nemotron 3.5 Super VL 120B total with 12B active per token, 512 routed experts per MoE layer, hybrid Mamba-2 and Transformer · DeepSeek V4 Pro 1.6T total with 49B active, text only.

• Context and output cap — 1,048,576 tokens of context for Super VL with no published output cap · 1,048,576 tokens of context for V4 Pro with 384,000 tokens of maximum output.

• Modality — text, image and video in with text out, through a C-RADIO vision encoder and a vision-language connector · text in, text out. No image path at all.

• Weights and licence — OpenMDW-1.1 on the NVIDIA checkpoint, commercial use permitted · DeepSeek publishes its own weights and permits commercial use as well.

• Price on a metered endpoint — none published; the model is self-host only · $0.66 per million input tokens and $1.98 per million output, with cache reads at $0.022.

• Release footprint — weights on Hugging Face 12 October 2026, vendor announcement 15 October 2026 · V4 Pro serving as the flagship, with the dated 0813 revision available for callers who need to pin.

Two of those lines decide most real deployments. The first is modality: if your input is a screenshot, a scanned contract, a chart or a video frame, half of this comparison is over before it starts, because DeepSeek V4 Pro has no image path and no announced plan for one. The second is the price line, and it is more interesting than it looks — NVIDIA publishes no rate for this model at all, which means the only way to know your cost is to know your token volume and your hardware bill.

Reading the table NVIDIA actually published

The flash-tier comparison is closer than the parameter counts suggest, and it is not uniformly unflattering to NVIDIA. On the rows where the two are nearest, the story is a genuine tie on knowledge and a real deficit on agentic execution.

• A statistical tie on graduate-level science — GPQA-Diamond 89.52 for Super VL against 89.6 for DeepSeek-V4-Flash. Four hundredths of a point separate a 120B model from a 1e12-class baseline on this row, both vendor-run.

• A deficit on the hardest reasoning — Humanity's Last Exam text-only 31.79 against 35.8, and MMLU-ProX 76.91 against 85.5.

• A deficit on competitive coding — LiveCodeBench v6 88.22 against 94.6.

• A deficit on agentic software work — SWE-bench Verified 71.10 against 75.4, Terminal-Bench 2.1 66.70 against 76.80, DeepSWE 24.65 against 42.3.

• A sharp deficit on long-context retrieval — AA-LCR v1.1 68.62 against 79.7. Both models advertise a million tokens; on the one independent-style recall row the flash tier is more than eleven points ahead. That is the single most consequential number on the table for anyone planning to use the advertised context rather than merely having it available.

• Rows where NVIDIA wins — GraphWalks 89.71 against 65.4, LMArena v3 proxy 52.67 against 49.8, and tau2-bench retail at 69.24 against 73.8 is close enough to be noise on a benchmark this noisy. AA-Omniscience is the more telling win: NVIDIA's hallucination rate of 30.0 against DeepSeek's 95.9, and a positive index of 4.03 against minus 7.8. A model that abstains is worth more than a model that asserts, and on this row the gap is not marginal.

• Agentic tools — tau2-bench airline 76.50 against 83.5 and telecom 73.19 against 98.2, both losses. On the telecom domain the flash tier nearly sweeps.

Every one of these is vendor-reported through NVIDIA's own NeMo Gym harness with a published recipe. That is a higher standard than a bare claim and a lower one than an independent rerun, and on this particular table the direction of most rows is against the model being announced, which is at least a sign the harness was not tuned to flatter.

What cannot be concluded about V4 Pro, and why it matters anyway

You cannot read any of the above as a prediction of the flagship matchup, for three reasons that are worth stating rather than glossing.

First, the baseline is not the flagship. DeepSeek's own published agent numbers for the 0813 revision of V4 Pro are on a different scale entirely: 87.9 on Terminal Bench 2.1, 83.3 on Cybergym, 74.1 on Toolathlon Verified, 62.7 on DeepSWE, 61.5 on NL2Repo, and 42.7 on Humanity's Last Exam without tools rising to 60.0 with them. Against NVIDIA's 66.70 on Terminal-Bench 2.1 and 31.79 on HLE text-only, the flagship gap is much wider than the flash-tier gap, and it is being widened by the vendor of the other model, which makes it no more trustworthy — only more pointed.

Second, the flash-tier column is a snapshot of a tier that has itself moved. A row that reads well today can read badly next month without either model changing, because the comparison depends on which checkpoint the vendor chose to run.

Third, the independent picture for the NVIDIA checkpoint is thin enough to be almost unhelpful as a tiebreaker: the third-party panel that scores these models has it in the mid-twenties on its index, with an unusually high output-token count per task and a note that the model is somewhat verbose. One data point, from one panel, four days after the weights appeared.

So the honest summary of the benchmark case is this: on vendor-run numbers, a 12B-active open vision model is roughly level with DeepSeek's cheap tier on knowledge and behind it on agentic execution and long-context recall; nothing in that table speaks to the flagship, and the flagship's own published numbers are better on every axis where the two can be compared at all.

The one axis where the comparison is trivial

Images. Not because Nemotron is good at vision — nobody has published an independent eval of its vision path, and the shipped runtime defaults sample at the card's text temperature rather than its vision temperature, which is a live trap for the first person to try it — but because DeepSeek V4 Pro does not accept them.

That asymmetry has a cost consequence that is easy to miss. A vision-capable hosted model bills a page of a scanned document by its token count, and a 300-dpi page is not a small number of tokens. A self-hosted 120B checkpoint bills the same page in GPU seconds. On a workload of a few thousand pages a month the difference is noise; on a few hundred thousand it is the entire budget conversation. NVIDIA's own vision rows — 62.82 on MMLongBench-Doc, 78.16 on MathVision, 80.82 on CharXiv reasoning, 68.12 on OCRBench v2 English — are vendor numbers, but they establish that the capability is present rather than nominal.

The other axis where the comparison flips is the one everybody asks about and few measure: memory. V4 Pro at 1.6 trillion total parameters does not fit anywhere near a single machine. Super VL at 82.3 GiB in its default 4-bit tag fits on one 96 GB card with room for a working context. That is not a quality argument. It is the difference between a model your team can pull and run tonight and a model you rent.

Running the arithmetic rather than the benchmark

Here is the calculation worth doing before reading another table. Take a workload of a million tokens of input and a hundred thousand tokens of output — a trivial month for a document pipeline, a heavy day for a chat product.

On DeepSeek V4 Pro's published rate, that is $0.66 of input plus $0.198 of output: under a dollar, before cache discounts bring the input side down further. On the NVIDIA checkpoint there is no rate, because there is no endpoint. The cost is whatever fraction of a GPU-hour that volume consumes, and at these volumes the answer rounds to nothing — but the fixed cost behind it does not. Four H100s do not become cheap because your first month's traffic is small.

That is the real shape of this matchup, and it is a volume question, not a quality question. Below a certain throughput the metered open-weight API is simply cheaper than the electricity and amortisation of the hardware, even though both models are open. Above a certain throughput the fixed cost inverts and the self-hosted checkpoint wins on cost regardless of which one scores higher on Terminal-Bench. The crossover depends on your utilisation, not on either vendor.

Where that leaves the routing question: if you would rather not run the cluster at all, DeepSeek V4 Pro is reachable on OrcaRouter at the provider's list price passed through with no per-token markup — $0.66 and $1.98, the same numbers DeepSeek publishes, because we do not add to them. A vendor price change is live on our side the same day rather than at the next contract renewal. Nemotron 3.5 Super is not on our key; we route a bit over two hundred models and this is not one of them, so if you need that checkpoint specifically, the vendor's own channels and the local pull are your route.

What to actually do

If your input is text and your workload is agentic — long tool chains, repository-scale edits, terminal sessions — DeepSeek V4 Pro is the better model and the cheaper one, and the NVIDIA checkpoint is not a serious contender against it on the vendor's own published rows. Run the workload on V4 Pro first, get a real cost per unit of work, and only then decide whether a cluster is warranted.

If your input is images, video or long documents, the comparison collapses into a single question about volume, because V4 Pro cannot take the input at all. Under a few thousand pages a month, keep it hosted. Above that, the 4-bit checkpoint on one 96 GB card is a credible piece of infrastructure — with two caveats worth repeating: the version most people will run has no published evaluation, and the runtime's own sampling defaults are tuned for the text half of what the vendor advertises as a vision model.

And if you do put a four-day-old checkpoint behind a request path, put a fallback behind it as well. The cheapest insurance available for a model with no operational history is a second model on the same call.