ZoomGain-9B vs MathForm 8B: Two 9B Checkpoints That Cannot Do Each Other's Job

A loaded question sits behind ZoomGain-9B and MathForm-8B: both are roughly nine-billion-parameter fine-tunes published inside the same eighty-day window, both are Apache-2.0-adjacent research releases from Chinese labs with a paper attached, both were post-trained by reinforcement learning on a few thousand examples, and both carry a benchmark score that looks impossible for their size. The temptation is to rank them. The problem is that one of them has never seen an image and the other has never seen a Lean proof, so the ranking has no axis.

ZoomGain-9B is inclusionAI's vision-language checkpoint from 10 November 2026, trained by GRPO on 9,000 questions mined from cropped image detail, and its strongest reported result is 90.25 on HR-Bench 4K. MathForm-8B is OpenBMB's autoformalizer from 14 August 2026, trained by supervised fine-tuning followed by reinforcement learning on roughly 367,000 Lean 4 examples, and its strongest reported result is an 88.06% syntax-check pass@8 average across six autoformalization benchmarks. Neither number says anything about the other model. What connects them is something more useful than a ranking: the two releases make unusually similar claims about how a small fine-tune can beat a much larger generalist, and the ways they justify those claims differ enough to tell you which one to trust for what.

The blank columns, named

ZoomGainBench contains no text-only model. It is a set of questions about photographs — attribute recognition, counting and OCR — so a model without a vision encoder cannot be entered, and MathForm-8B was not. Conversely, MathForm's six evaluation benchmarks are FormalMATH, ProverBench, CombiBench and three FATE splits, all natural-language-mathematics-to-Lean-4 translation tasks. ZoomGain-9B has never been run on any of them and there is no reason to believe it would do well, since nothing in its training involved formal languages.

This is not a technicality about benchmarks. It is the entire shape of the decision. If your task produces an image and consumes a short factual answer, MathForm-8B is a text model that cannot attempt it. If your task consumes a mathematical statement and produces a machine-checkable Lean 4 proof, ZoomGain-9B is a vision model that has never been asked. There is no overlap to measure, which means the honest comparison is of construction, evidence and operational cost — and those turn out to differ a great deal.

Two RL recipes, both small, and what differs between them

ZoomGain's training set is 9,000 records over 7,713 source images. One epoch, batch size 64, eight rollouts per question, GRPO with a reward that checks whether the selected option matches the reference answer. The data was generated, not annotated: several models locate a region, crops are captioned and compared with full-scene captions, and details the crop reveals become questions about the full image. Human involvement was screening only — the paper is explicit that no manual question-and-answer editing took place.

MathForm's training set is FormalVerse, roughly 367,000 verified Lean 4 examples. Supervised fine-tuning first, then reinforcement learning with feedback from Lean compilation and a semantic-consistency check. Data construction used Mathlib retrieval before generation and iterative refinement after it, with compiler diagnostics steering the revisions.

• Base model — ZoomGain-9B is post-trained from Qwen3.5-9B (32 layers, hidden 4,096, intermediate 12,288, 262,144 positions, a 3:1 linear-to-full attention schedule). MathForm-8B is post-trained from Qwen3-8B (36 layers, all full attention, hidden 4,096, intermediate 12,288, 40,960 positions, 32 query heads against 8 KV heads at head dimension 128, rope theta 1e6).

• Shared dimensions — both inherit a 4,096 hidden width and a 12,288 intermediate width from their Q​wen parents, which is the single most striking coincidence between them and is a property of the family, not of the fine-tuning.

• Vocabulary — 248,320 tokens for ZoomGain-9B against 151,936 for MathForm-8B. MathForm's is a Qwen3-generation vocabulary and it matters: Lean 4 code, Unicode mathematical symbols and proof tokens tokenize differently in each.

• Context — ZoomGain-9B's card states nothing and its config declares 262,144; MathForm-8B's config declares 40,960, and its recommended serving command is capped at 16,384 with a 16,384-token max_new_tokens. For autoformalization, output length is the binding constraint, not input.

• Modality — image and text in, text out for one; text in, text out for the other.

• Licence — ZoomGain-9B: code Apache 2.0, checkpoint terms deferred and no licence tag in Hub metadata. MathForm-8B: Apache 2.0 with a LICENSE file in the repository.

• Distribution — ZoomGain-9B has BF16 plus GGUF at F16, Q8_0 and Q4_K_M. MathForm-8B has BF16 safetensors in four shards totalling about 16.4 GB plus community GGUF and MLX builds, but no vendor-provided quantisation of its own.

MathForm-8B's numbers, and why they are easier to check than they look

OpenBMB reports an 88.06% average syntax-check pass@8 and 72.37% consistency-check pass@8 across its six benchmarks. On the same table, ReForm-8B reaches 81.76 and 66.21, Goedel-Formalizer-V2-32B reaches 78.28 and 63.74, and StepFun-Formalizer-32B reaches 63.65 and 44.47. The three FATE splits are where the specialised models collapse and MathForm does not: FATE-X lands at 54.00 syntax and 37.00 consistency against ReForm-8B's 45.00 and 21.00.

Two things make those figures more credible than a typical vendor table. First, the tasks are mechanical in a way perception tasks are not — a Lean 4 statement either compiles or does not, and the consistency check compares meaning against the source proposition. A benchmark that is partly machine-verifiable is harder to score optimistically. Second, the paper states that the SFT-only model is worse: MathForm-8B-SFT scores 84.38 syntax and 66.53 consistency, so the RL stage is doing measurable work rather than decorating a finished model.

There is also a comparison worth quoting because it cuts against the model's own framing. On the same six benchmarks, Qwen3.7-Plus — a general-purpose commercial endpoint — scores 86.33 syntax and 83.38 consistency. MathForm-8B beats it on syntax by 1.7 points and loses on consistency by 11 points. The specialised fine-tune wins the exact-match task and loses the one that requires the formalization to mean the same thing as the source. If your pipeline needs a statement that compiles, a small specialist is a bargain. If it needs one that is faithful, a general model with retrieval may still be ahead.

ZoomGain-9B's numbers, with the same accounting

The authors report 90.25 on HR-Bench 4K against 82.13 for the base, 93.19 on V*Bench against 86.91, 56.31 on VisualProbe against 44.66, 70.00 on MMVP against 60.00 and 68.12 on O3-Bench against 56.52. The held-out average across thirteen benchmarks is 66.10 against 59.75. On ZoomGainBench — the benchmark built by the same pipeline as the training data — the model scores 55.48, and the paper labels that column "Held-in" rather than burying the distinction.

The structural difference from MathForm's evidence is that ZoomGain's benchmark is not machine-verifiable in the same way. Multiple-choice answers are exact-match, counting is exact-match on an integer, and OCR is ANLS-scored — but the questions themselves are generated, and the authors' own check is that the answers can be verified on the cropped view. A generated question with a verified answer is still a generated question.

Both releases were produced partly with models as annotators. ZoomGain's appendix names the roles: Qwen3.5-397B-A17B for scene captioning, grounding and final MCQ verification; Qwen3-VL-235B-A22B-Instruct and Kimi-K2.6 for crop-based verification and blind answering; DeepSeek-V4-Pro for distractor clustering. MathForm's FormalVerse was built by a pipeline that retrieves from Mathlib and then lets the compiler reject bad generations. One pipeline leans on a verifier that cannot be talked into agreement; the other leans on models checking other models' work, with human screening only.

Serving them, in the terms your ops team will ask about

ZoomGain-9B is 18.82 GB in BF16 and runs under vLLM 0.19.1 with transformers 5.7.0 — the authors' exact versions — or under a llama.cpp build with Q​wen3.5 multimodal support using the GGUF files plus a projector. Its linear-attention-heavy trunk keeps the KV cache modest at long sequence lengths, which is the property that matters when a single request carries a high-resolution image rather than a long document.

MathForm-8B is about 16.4 GB in BF16, served by vLLM with a 16,384 max model length or by SGLang with the same context, at temperature 0.6 and top-p 0.95 with a 16,384-token output budget. Its evaluation requires a running Kimina Lean Server and Lean 4.21.0, which is the part people underestimate: you are not just deploying a model, you are deploying a Lean toolchain next to it. Compilation feedback is the reward signal during training and the acceptance test during evaluation, so a pipeline without Lean cannot measure whether this model is working.

Neither is in the OrcaRouter catalogue today, and this piece does not claim otherwise. What we can do is route the surrounding work. A document pipeline that reads a scanned page and then checks the mathematics on it has two halves: the vision call and the reasoning call. One key across 200-plus models at provider list price with no markup added, a routing rule that sends each half to an endpoint that clears your bar, and automatic failover if either stalls — that is the layer underneath both halves, and it is why a two-model pipeline does not have to mean two vendors and two contracts.

Which one to download

Pick MathForm-8B if your problem is formal and your toolchain is already Lean. The evidence is stronger, the licence is unambiguous, the output is machine-checkable, and the 11-point consistency deficit against a general model is a real limitation you should know about before you commit. Pick it for the FATE splits especially, where the specialised baselines fall to 21% consistency and it holds 37%.

Pick ZoomGain-9B if your problem is a detail inside a picture and your answer is short. Its held-out gains are believable for 9,000 samples, its benchmark is downloadable so you can check the claim yourself, and its licence is the one thing you must read before shipping. Do not pick it expecting a general vision assistant; nothing in the release tests that.

And if you were hoping for a verdict on which is the better nine-billion-parameter model, the release notes answer it better than any ranking could: they were built to be checked against different kinds of ground truth, and only one of those ground truths compiles.