AesCode-8B vs MathForm-8B: Both Are 8B Fine-Tunes Whose Output a Machine Can Check

AesCode-8B and MathForm-8B landed within eight weeks of each other, both from repositories rather than press releases, and the coincidence is more interesting than it first looks. Both start from a Qwen3-family checkpoint. Both spend their entire training budget on a narrow output shape. And both were built around a checker: MathForm-8B is trained against a Lean 4 compiler's verdict, and AesCode-8B is scored by rendering each candidate page in a sandboxed browser and reading back the DOM, the computed styles and a screenshot. Neither is a chatbot and neither is trying to become one. What separates them is what a machine can verify and what it cannot — and, in the case of the newer of the two, what happens when half the score arrives from a judge nobody has named.

The publication records are not symmetrical. MathForm-8B came from OpenBMB and its model card dates the release 2026-08-14; it is built on Qwen3-8B and trained on FormalVerse, a corpus of roughly 367,000 verified Lean 4 examples, with supervised fine-tuning followed by reinforcement learning that uses Lean compilation and semantic-consistency checks as its reward signal. AesCode-8B carries no release date anywhere in its files. Microsoft created the Hugging Face repository on 2026-09-29, committed the weights at 03:35 UTC on 2026-10-07 under the message "Release AesCode-8B", and published the training code on GitHub on 2026-10-08. No announcement accompanied either event, the model card's citation reads "Under review, 2027", and the repository showed two downloads as of this writing. It is fine-tuned from Qwen3-VL-8B-Instruct, which is worth noting precisely because it is not the same ancestor as MathForm-8B's.

The ancestry explains most of the split

Qwen3-8B and Qwen3-VL-8B-Instruct share a generation and a family name but not a job. Qwen3-8B is a text-only generalist: roughly 8.2 billion total parameters, about 7 billion of them non-embedding, grouped-query attention, a 32K-token native context extensible to 131K through YaRN, and training across 119 languages and dialects. Qwen3-VL-8B-Instruct is the vision-language sibling, and it is the checkpoint AesCode-8B starts from — the published AesCode config is a straight Qwen3-VL recipe with 36 hidden layers, hidden size 4,096, 32 attention heads with 8 key-value heads and a 151,936-token vocabulary.

That fork decides the input side of both specialists before either was trained. MathForm-8B takes text and emits text in a formal syntax. AesCode-8B takes text plus an optional reference image and emits a document.

• Base — MathForm-8B: Qwen3-8B, text-only. AesCode-8B: Qwen3-VL-8B-Instruct, image and text in.

• Parameters — MathForm-8B: about 8.2B. AesCode-8B: about 8.8B in bf16 across four shards, which Hugging Face rounds to 9B.

• Training data — MathForm-8B: FormalVerse, roughly 367K verified Lean 4 examples. AesCode-8B: 3,000 cold-start demonstrations, then GDPO reinforcement learning over 7,408 prompts for 400 steps.

• What checks the output — MathForm-8B: a Lean 4 compiler, plus a semantic-consistency check against the original problem. AesCode-8B: a sandboxed Playwright render with six deterministic verifiers and one model-scored rubric.

• Licence — both Apache 2.0, both ungated, both inheriting from the Qwen3-family backbone.

• Hosted anywhere — neither, as far as we can find.

Two different meanings of "verifiable"

This is the distinction worth being slow about, because "machine-checkable" gets used for both and it does not mean the same thing.

MathForm-8B's checker is a proof assistant. Lean 4 either accepts a statement or it does not, and the verdict is not a matter of opinion, a rubric or a judge's taste. The training loop is pointed at that signal: the SFT stage on FormalVerse teaches the mapping from an informal problem to a formal theorem statement with an imports header and a named theorem, and the RL stage sharpens it using compilation plus a consistency check that asks whether the formalization still says what the original problem said. Compilation is binary and reproducible by anyone with the same Lean version. Consistency checking is the softer half, and it is the half where the reported numbers get weak — which is exactly what the published results show.

AesCode-8B's checker is a renderer. Candidates are rendered in a sandboxed Playwright browser with external requests blocked, and the harness reads back the DOM, computed styles, bounding boxes, console status and a screenshot. Six deterministic channels score the parsable things — execution, exact text, boundary behaviour, table and chart data, semantic layout, whitespace — and a seventh, the Visual Graph Rubric, scores geometry and placement through graph-bound yes/no questions. Tables must be real HTML tables and charts must be ECharts specs, which is a constraint doing real work: it forces the output into a shape a verifier can parse. The deterministic half is genuinely reproducible. The visual half is judged by a vision-language model whose identity the documentation does not name, which means nobody outside the lab can reproduce it.

So the honest comparison is not "one is verified and one is not." It is that MathForm-8B's primary signal is a compiler and its secondary signal is a consistency check, while AesCode-8B's primary signal is a set of deterministic DOM assertions and its secondary signal is a model's opinion, packaged inside the same overall score.

What each one reports, and what that is worth

MathForm-8B reports an average Pass@8 of 88.06% under the syntax check and 72.37% under the consistency check across six benchmarks. The per-benchmark spread is the interesting part: 95.06% consistency on FormalIMATH and 94.83% on ProverBench, then 63% on FATE-H and 37% on FATE-X. Those last two are the hard, realistic statements, and the fall from the mid-nineties to the mid-thirties is the honest shape of the capability. All of these figures are vendor-reported and unreproduced, and the benchmark mix is weighted toward the easier sets.

AesCode-8B reports 82.94 Overall on Microsoft's 300-sample infographic rubric — Text 94.06, Boundary 88.36, Chart 87.79, Rule 90.07, Content 86.41, Layout 87.80, Style 53.21, Visual 75.80 — with three generations per prompt and no selection. Microsoft also reports that it beats reference-conditioned GPT-5.5 at 81.28 and Claude Opus 4.8 at 80.39 on the same rubric, that a severe canvas-overflow failure recurs on 4.3% of the 300 samples, and that 22.4 Visual points separate the 32B companion from its own backbone. Every number is the vendor's, on the vendor's task, scored against channels the vendor designed.

The two sets of numbers cannot be compared to each other at all. There is no shared task, no shared metric and no shared judge. Putting 88.06% next to 82.94 would be comparing a Lean formalization pass rate to an infographic overall score, and neither model was ever evaluated on what the other does.

One asymmetry is worth naming because it cuts against the newer model. MathForm-8B's headline metric has an external referee built in: anyone can install Lean, load the same benchmarks and check whether the statements compile. AesCode-8B's headline metric does not — the deterministic verifiers could be re-run by a determined outsider, but the visual half of the score depends on a judge the paper has not identified. An unreproduced compiler pass rate is a weaker claim than a benchmark table, and still a stronger one than an unreproduced rubric score with an anonymous grader inside it.

Running them is a different question from either score

Both are self-host decisions today. MathForm-8B is the cheaper one by a wide margin: a roughly 8.2B text-only checkpoint with a generation budget around 16K tokens of Lean output, which quantises onto a single mid-range card. AesCode-8B is an 8.8B vision-language model whose serving path carries images as well as text; the card's own command is vllm serve microsoft/AesCode-8B --limit-mm-per-prompt image=2 --max-model-len 24576, and 17.5 GB of bf16 weights plus a KV cache for 24,576 tokens and two images means a 24 GB card is tight and 40-48 GB is the realistic floor. Budget a rendering stack as well if you want to score your own outputs, because that is how every quality claim about the model was made.

The bigger hidden cost is that both models are specialists you would be adopting permanently. A team that needs formalization and document generation now runs two 8B serving paths, two sets of prompt formats, two failure profiles, and neither model can absorb the work of the other. That is the case a routing layer exists for: keep the specialists where the economics and the data handling justify owning a GPU, and send the general traffic to something hosted behind the same endpoint. Concretely, the generalist siblings of these two bases are callable — Qwen3-VL-8B-Instruct at $0.18 per million input and $0.70 per million output over a 131,072-token context, alongside the Qwen 3.8 family and other open checkpoints — all through OrcaRouter's one API covering 200-plus models, with provider list price passed through at 0% markup and automatic failover between providers. Neither specialist is routable here or anywhere else we can find; what is routable is the generalist you fall back to when the narrow job is finished, which is the difference between trialling a research checkpoint and making it a load-bearing dependency.

Choosing between them, if you really have to

Pick MathForm-8B when the artifact has to compile. Problem-bank conversion, formal corpora for a prover, pre-formatting statements for Lean-based tooling — that is the entire job description, and it is the only one of the two that was trained for it. Take the FATE numbers seriously when you scope: on the hardest realistic statements, roughly a third come out consistent, and you will be building a human review step regardless.

Pick AesCode-8B when the artifact has to render. A brief goes in, an editable HTML document comes out, tables are tables and charts are chart specs, and the whole thing diffs in Git. Accept the Style ceiling — 53.21, a dimension defined as needing no further visual revision before delivery — as the honest measure of how much editing remains, and accept that the 24,576-token context has only been validated on single infographic pages rather than the multi-slide decks people actually want.

The choice most teams will actually face is neither of these, though. It is whether one of these narrow specialists is worth a deployment at all, or whether the generalist behind it, called over an API, is close enough for the volume you have. That is an afternoon of prompt testing rather than a GPU purchase, and both cards' own numbers give you the reason to run it: MathForm-8B's hard-set consistency sits at 37%, and AesCode-8B's Style score sits at 53%, so neither is a model you would put in a pipeline unsupervised.

What both releases tell you about how models ship now

Two 8B fine-tunes, eight weeks apart, from two different labs, released with no announcement, no product page and no independent evaluation, both built around a verification loop, both Apache 2.0, neither served by anyone. That pattern is the story more than either model is. The research method has moved into the reward function — OpenBMB's compiler signal, Microsoft's decoupled cross-modal channels — and the published artifacts have become the training recipe plus the weights, with the paper arriving later, if at all.

What that means for anyone reading a comparison like this one is that the vendor's own numbers are all you get for a while, and the useful question is not how high they are but how checkable they are. AesCode-8B's overflow rate and its Style ceiling are checkable claims dressed as failures. MathForm-8B's FATE-X consistency figure is the same thing. Those are the numbers to read, and the ones to go back and re-run yourself the moment the checkers are reproducible end to end.