Inkling 2 Small Weights Are Down: 394 GB, Apache 2.0, and a Node-Sized Bill

The first weight commit in the Inkling 2 Small repository is stamped 19:53 UTC on 12 November 2026 — nine hours before this was written. Until then the model was a rate card and a benchmark table: Thinking Machines Lab launched it on 5 November with closed weights, a Tinker endpoint and a stated intention to publish the checkpoints under Apache 2.0 "next week." The model itself is therefore eight days old. What is new is that you can download it, and the download is not a small file.

Two repositories are public. The primary checkpoint is 393.5 GB across 36 files, 24 of them safetensors shards. The MXFP8 variant is 622.8 GB. Both are declared Apache 2.0 in the model card metadata. That is the whole of the announcement so far — the model card, the repositories, and a one-paragraph update appended to the launch post. Everything below is read off those primary sources plus the two serving cookbooks that shipped alongside them, and it is worth being blunt about the thing that is still missing: the external safety testing that was the stated precondition for opening these weights has not been published.

What is actually in the download

Start with the size, because it is the first thing that decides whether this model is available to you at all. The mixed-precision layout explains the number almost exactly, which is a good sign that the published figures are consistent with each other rather than marketing copy.

• Checkpoint split — NVFP4 experts in layers 0 through 39, MXFP8 experts in layers 40 through 47, per the SGLang cookbook.

• Predicted size — averaging 5.33 bits per parameter over that 40/48 and 8/48 split gives 396 GB for 594B parameters.

• Measured size — 393.5 GB for the primary repository, including tokenizer and config files.

• MXFP8 repository — 622.8 GB, and the safetensors index counts 612.4B stored parameters against the primary repository's 383.1B, because packing expert weights to four bits reduces the number of counted elements rather than the model.

The architecture underneath is a 48-layer decoder-only transformer with a sparse mixture-of-experts feed-forward block: 384 routed experts, 8 active per token, 4,608 hidden width, 48 attention heads over 8 key-value heads at 128 dimensions each, and a 201,024-token vocabulary. Attention is hybrid rather than uniform — 40 of the 48 layers use a 1,024-token sliding window and only 8 are global, which is where the long-context budget comes from. The vision tower is a separate 23-layer transformer with 1,024 hidden width and 16 heads, reading 20×20-pixel patches with a 2×2 merge, so each image token covers a 40×40-pixel region; the card asks for images between 40 and 4,096 pixels on a side.

There is also a multi-token-prediction head bolted on for speculative decoding — five layers, a block size of 8, a Markov rank of 256, and target layers at 5, 11, 17, 23, 29, 35, 41 and the final block. Those are precisely the eight global-attention layers, which is the detail that tells you the draft head was designed around the hybrid stack rather than added after the fact. Thinking Machines calls the resulting decoder "DSpark."

One housekeeping item for anyone whose legal review asks for paperwork: the licence is declared in the model card metadata as Apache 2.0 and the card links out to the standard deed, but there is no LICENSE, LICENSE.md, LICENSE.txt or NOTICE file in the repository tree. All four return nothing. The declaration is real and it is unambiguous; the artefact most legal teams expect to find is not there.

The hardware list, and where the vendor and the cookbook disagree

The model card gives a VRAM floor. The serving recipes give a topology. They are not the same number, and the gap between them is the most practical thing on this page.

• NVFP4 primary — at least 500 GB of aggregated VRAM; the card's examples are 2× B300 or 4× H200.

• MXFP8 variant — at least 800 GB; examples are 4× B300 or 8× H200.

• SGLang, per-GPU topology — GB200, GB300 and B300 at TP 4 / EP 4; B200 at TP 4 / EP 4 with TP 8 recommended for KV headroom; H200 with NVFP4 at TP 8 / EP 8; MI350X and MI355X at TP 8 / EP 8; Vera Rubin at TP 4 / EP 4; TPU7x at 8 / 8 on a four-chip host.

• MXFP8 on B200 or GB200 — TP 8 / EP 8, roughly 151 GB of weights per GPU.

• The constraint behind all of it — expert parallelism must equal tensor parallelism, because NVFP4 expert activation scales span each expert's full input width and cannot be tensor-parallel sharded.

So the card's 4× H200 example meets the 500 GB floor but is not a topology the cookbook validates; H200 is listed at 8-way. Storage and memory floors tell you whether it fits. They do not tell you whether it runs, and for this checkpoint the second number is twice the first on the hardware most people have. If you are provisioning, plan against the recipe, not the floor.

The AMD path is two days old and not merged

The radar that surfaced this piece caught the AMD support pull request rather than the weight release, and the honest framing of that PR on its own is that it is engine work — the kind of thing that is not news. It matters here only because it landed in the same week the checkpoints did, and because it is the second silicon vendor with a container image.

The pull request adds gfx950 support for MI350X and MI355X, stacked on the base Inkling 2 Small port. What it changes: the ROCm kernels live in separate *_hip.py modules reached only under HIP, CUDA and NPU paths are untouched, the unified KV pool and full-graph prefill become launch defaults, MegaMoE is refused rather than attempted, and quick-reduce quantisation is ignored unless an environment flag allows it. gfx950 is required unless a second environment variable overrides it — meaning MI300-class cards are refused at launch, and the PR author says gfx942 "has not been run."

The measurements in the PR, which are the author's own and unreviewed: 426 unit tests passing on one MI350X, 98.0% on GSM8K-200 at balanced effort and 97.5% with DSpark at an acceptance length of 4.04, and time per output token at 2,048 in and 512 out of 5.15 ms balanced against 2.10 ms with DSpark. Those convert to roughly 194 and 476 output tokens per second on a single stream across eight accelerators. With AITer enabled the accuracy figures move to 97.5% and 96.5%.

The state of the PR is draft. CI has not run on it — the base job is blocked on a missing label and there is no AMD run recorded against the commit under review — and all listed code owners are still awaiting review. Treat MI350X support as something you can build from source this week, not as something you can buy.

Serving it means a dev image, not a release

Both major stacks have day-zero support and neither has shipped it. SGLang says plainly that support is in no release yet and points at dev-inkling-2-small builds, with separate images for Hopper and Blackwell, Vera Rubin, MI35x and TPU7x. vLLM asks for a nightly or newer, or the vllm/vllm-openai:inkling-2-small image. That is normal for a model this size and this new, but it changes what "supported" means: you are tracking a branch, and a rebase can break your deployment.

The operational details that will actually bite:

• Context — vLLM's TP 4 recipe caps at 262,144 tokens on one tray; the full 1,048,576-token window needs TP 8 across two nodes. At TP 4, a single 1M-token request needs 64.47 GiB of KV cache, which is more than a tray has left.

• Speculative decoding — SGLang verifies 8 tokens per step with the bundled DSpark head; vLLM's equivalent sets 7 speculative tokens with probabilistic drafting. The recipe warns that acceptance drops 15 to 25% at temperature 1.0 if you draft greedily.

• Parsers — reasoning uses inkling, tool calls use inkling2, and SGLang selects both automatically when unset. vLLM's tool-call parser is opt-in and the recipe says tool calling has not been benchmarked end to end, only unit-tested.

• Prefill/decode disaggregation — validated only as 1P1D on a four-GPU GB300 tray, NVFP4, no MegaMoE, with static memory fraction forced down to 0.86 because the automatic 0.963 value ran out of memory. The split does not fit the MXFP8 checkpoint.

• Encoder disaggregation — not in the images yet, and the language server stops if no healthy encoder is reachable.

• Memory headroom — on 192 GB B200 or GB200 parts at TP 4, KV capacity is about a third of what a GB300 tray gives you, which is the real limit on long-context concurrency rather than the window size printed on the card.

There is also one community report on the model card itself, from the first hours after upload, titled "Hopper cards not currently compatible with the vllm image." No maintainer reply is visible and I could not reproduce it, so treat it as a flag rather than a finding, and check before you commit a Hopper node. The SGLang side does list H200 at 8-way, so the Hopper path is not closed — it is just not the path vLLM's recipe describes.

What self-hosting costs, and why the answer is rarely price

The reason to download 394 GB is control, not unit economics, and the arithmetic makes that clear quickly.

On Tinker the model lists at $0.30 per million input tokens, $0.03 per million cached input, and $1.00 per million output, at 256K context. Artificial Analysis measures 84 output tokens per second and 67.71 seconds to the first answer token, at $1.18 per index task and 29.6 minutes per task, against 154M output tokens across the suite and a peer median of 89.8M. That token count is the tell: this is a model that thinks at length before it answers, and at list output pricing a single stream running flat out for an hour produces about 302,000 tokens, or roughly thirty cents of output. To beat a node that costs a dollar an hour you need to keep about four streams busy; to beat a twenty-dollar-an-hour node you need sixty-six. Very few evaluation workloads, and even fewer agentic ones, hold that concurrency against a model with a 67-second time to first token.

What you buy by owning the weights instead: the full 1M window on your own terms rather than at Tinker's 256K serving tier, no beta waitlist in front of production, fine-tuning on your own stack, data that never leaves your network, and eval sweeps that are not metered per token. What you give up: a working release build, the eight-way topology you have to buy rather than rent, and a serving stack you now maintain.

That is the honest shape of the decision, and it is also where a routing layer fits without being asked to pretend it hosts something it does not. Inkling 2 Small is not in the OrcaRouter catalogue — we checked the live model list this week and there is no inkling entry among 215 — so the API path for this model stays Thinking Machines' own Tinker platform and, per the launch update, several third-party inference providers. What more than 200 models behind one key does give you is a fallback: you can run the self-host experiment on a single node and keep production on models we do route, with automatic failover if the experiment falls over, and on the models we do route provider list price is passed through unchanged. When a second and third host picks up these checkpoints, the failover leg stops being theoretical — and the vendor lock that is the main argument for downloading 394 GB in the first place gets weaker.

Where it sits now that it is open

The score did not change when the weights landed; what changed is that it now belongs to a leaderboard it was previously excluded from. On Artificial Analysis' open-weights ranking Inkling 2 Small is fourth of 28 at an Intelligence Index of 31, against a comparison-set median of 24.

• Kimi K3.1 Pro — 46.

• DeepSeek V4.1 Flash 1017 at max effort — 33.

• GLM-5.3 at max effort — 32.

• Inkling 2 Small — 31.

• MiMo-V2.6-Pro and Ling 3.1 Flash 1019 — 30 apiece.

• Mistral Large 4 at high effort — 26.

• Qwen3.8 27B at xhigh — 20.

The interesting column in that list is not the top. Every model above Inkling 2 Small is from a Chinese lab, and the best open-weights model from a non-Chinese lab was Mistral Large 4 at 26. Opening these weights closes that particular gap from twenty points to fifteen and moves the top of the non-Chinese column. It does not touch the frontier: the best open model is fifteen points clear, and closed models sit further out still.

The vendor's own evaluation table is longer and more flattering, and every figure in it was run by Thinking Machines on its own harness at high effort — unreproduced, and therefore a ceiling rather than an expectation. The rows that matter for agentic work: SWE-bench Pro v2 Hard 86.3%, DeepSWE v1.1 72.1%, Vibe Code Bench v1.1 81.6%, BrowseComp 90.2% and DeepSearchQA 90.9% both with compaction, MCP Atlas 85.4% full and 85.0% public, HLE 44.1% text-only, FrontierCode v2.0 25.4%. The row that cuts the other way is Terminal Bench Science at 5.7%, which is low in absolute terms and consistent with the long-horizon, long-deliberation profile rather than a broad capability claim. The vendor prints 30.6 for the Intelligence Index where the live board shows 31 — a snapshot difference, not a dispute.

What would change this assessment

Three things, in order of how much they matter.

The safety process that was supposed to gate this release has not been published. Thinking Machines' July post on a safe path to open weights describes four external organisations doing pre-deployment testing on the first-generation Inkling, and the 5 November launch post tied this release to "external safety testing and red-teaming." The 12 November update announcing the weights contains one sentence and one link — to the Hugging Face repository — and no evaluation results, no red-team findings and no revision to the earlier conditions. That does not mean the testing did not happen. It means a reader cannot check, and an organisation with a procurement policy that asks for the evidence will not find it here.

Release-grade support would change who can adopt it. Right now the choice is a dev branch in SGLang, a nightly in vLLM, or building the AMD path from source off an unmerged draft. The moment support lands in a tagged release, the model becomes a deployment decision rather than an engineering project, and the eight-way topology stops being a research setup.

Independent reproduction at default settings would tell you what 31 is worth in practice. The agentic rows are the ones to watch — SWE-bench Pro at 86.3 and DeepSWE at 72.1 are strong claims for a model with 17B active parameters, and they were produced at maximum effort. Third-party runs at default effort land lower, always. The question is by how much, and nobody outside Thinking Machines has answered it yet.

Until at least one of those lands, the correct description of Inkling 2 Small is unusually precise: an Apache 2.0 checkpoint you can download today, that no released serving stack supports, whose strongest claims come from the lab that made it, and whose licence gate opened without the paperwork that was supposed to precede it. That is a real release and a real gap, and the gap is the part worth watching.