Clef vs MathForm-8B: One Answers Your Question, the Other Writes a Proof

The reason to put Cloudflare/clef next to openbmb/MathForm-8B is that they are the two most easily confused things in open weights right now, and confusing them costs a training run. Both are fine-tunes built on the Qwen3.8-27B and Qwen3-8B checkpoints respectively. Both are Apache-2.0. Both were published within the last ten weeks. Both are narrow, purpose-built, and described by their makers as specialised rather than general. And they have nothing whatsoever to do with each other, because one produces a number selected from a list you supplied and the other produces Lean 4 source code, token by token, for a proof assistant to check.

Clef is Cloudflare's 27-billion-parameter multimodal decision model: you give it a state and a schema of typed questions, and it returns a calibrated probability for every allowed answer in a single non-autoregressive pass, with no text generation anywhere in the path. MathForm-8B, from OpenBMB, is an autoformalisation model — a natural-language mathematical statement goes in, Lean 4 comes out, and the pipeline that trained it is described in an arXiv preprint published alongside the weights on 14 August 2026. One model's ceiling is the size of your option list. The other's ceiling is the strength of Lean's type theory. Asking which is better is a category error, and the fact that both are eight-to-twenty-seven billion parameters of Apache-2.0 open-weights fine-tune is exactly what makes the error easy to commit.

What each model's interface physically forbids

The fastest way to see the split is to read what each one can return.

• Input — Clef takes a state (text, JSON, images or video frames) plus 1 to 64 named typed questions; MathForm-8B takes a natural-language mathematical statement.

• Output — Clef returns one probability per allowed option, softmaxed per question. MathForm-8B returns Lean 4 source code.

• Question types — Clef's are noul (true/false, with the probability of true), choice (2 to 26 named options) and score (2 to 26 ordered levels); MathForm-8B has no question concept at all.

• Calibration — Clef publishes a confidence field per answer; MathForm-8B publishes Pass@8 rates under syntax and consistency checks.

• Serving — Clef is served through a Jev/SystemOne-compatible POST /v1/systemone body, hosted or self-run; MathForm-8B ships with Transformers, vLLM and SGLang instructions, all exposing an OpenAI-compatible completion endpoint at a 16,384-token context with max_new_tokens set to 16,384.

The practical consequence falls out immediately. If you need a yes/no judgement about a piece of text, Clef is the only one of the two that can produce it — there is no way to ask MathForm-8B a question that is not "write the Lean 4 for this." If you need Lean 4, Clef is the only one of the two that categorically cannot produce it, and not through weakness: asking a schema-bound scorer to formalise a theorem does not fit its contract, so the request is refused by construction rather than answered badly.

The pipeline where both of them belong

There is a real architecture in which these two models sit next to each other, and it is worth walking through because it makes the split concrete rather than abstract. Consider a service that formalises mathematics for researchers and students.

Statements arrive in natural language, unbounded in kind. Before anything can be formalised, something has to decide what arrived. Is this a theorem to prove, a definition to add, a request to check an existing proof, or a question that needs clarification before anyone touches Lean? Is it self-contained, or does it lean on context the user did not supply? Are the symbols conventional, or is this notation the system has never seen? Every one of those is a bounded question with a small option set — Clef's exact shape — and the calibrated probability attached to each answer is what lets the service do something sensible with the uncertain ones: route anything below a threshold to a human instead of guessing.

The second step belongs to MathForm-8B. Take a statement that has already been classified as a self-contained theorem with known notation and emit the Lean 4. That is a generation problem, and the quality question is how often the output compiles and how often it means the same thing as the original — which is precisely what the vendor's two evaluation axes measure. This is the general pattern worth extracting from the pairing: a cheap bounded classifier in front of an expensive specialised generator is usually cheaper and more reliable than prompting the generator to decide whether it should be running at all.

What the numbers on each side actually measure

OpenBMB's arXiv abstract reports that MathForm-8B reaches average Pass@8 rates of 88.06% under Syntax Check and 72.37% under Consistency Check across six benchmarks, and that on the FATE-H and FATE-X subsets it achieves consistency pass rates of 63% and 37%, both above the strongest specialised baselines the paper compares against. The training pipeline is the interesting part of the claim: a retrieval planner pulls relevant definitions and existing formalisations out of Mathlib before generation, and generated statements are then revised using compiler diagnostics and semantic-consistency feedback, which is how the FormalVerse dataset of roughly 367,000 verified Lean 4 examples was built before supervised fine-tuning and reinforcement learning. These are vendor-reported figures from a paper and a model card. Nobody independent has rerun them.

Clef's numbers measure something else entirely and are not comparable. Cloudflare's Decision Index run reports BANKING77 intent macro-F1 of 94.2, CLINC150 with out-of-scope handling at 97.4, GPQA Diamond at 48.0 — where the older Jev scores 78.3 — and a median request latency of 209.3 ms. It is a table about classification and routing, produced by the vendor on the vendor's own suite, hosted on the vendor's own leaderboard, and unreproduced.

The one honest sentence to write about these two columns is that they share no benchmark, no unit and no evaluation philosophy. MathForm-8B's 37% on a hard formalisation subset and Clef's 97.4 on out-of-scope intent detection are both real claims by their makers about different jobs, and a reader who lines them up has learned nothing except that both numbers exist.

What you would run, and what it costs

Neither model is a route on OrcaRouter, and this article makes no availability claim — the catalogue returns a 404 for cloudflare/clef and for MathForm-8B. The MathForm repository measures about 16.4 GB, and both models are Apache-2.0, so both can be self-hosted tomorrow by anyone with the hardware.

• Clef on Cloudflare — $0.24 per million input tokens on Workers AI, with the published latency measured on a single H200.

• MathForm-8B self-hosted — no vendor hosting, no per-token price, and a documented 16,384-token context, which is a constraint worth noticing: a long paper section will not fit in one pass.

• The routed alternative for the classifier half — TypeSafe's Jev 1.13 at $0.042 per million input tokens over a 65,536-token context, served through the same POST /v1/systemone body Clef uses, which makes it a drop-in for the first step of the pipeline above.

That is where a routing layer earns its place in this particular pairing. The pattern is a decider and a generator, and the decider half is the one with a live substitute — one endpoint in front of 200-plus models, provider list price passed through at no per-token markup, and automatic failover so a provider's bad afternoon does not stall the front door of your pipeline. The generator half is a 16.4 GB artifact you run yourself, and no endpoint changes that.

One more thing the sizes hide

The parameter counts invite a comparison that does not hold. Clef is 27B and MathForm-8B is 8B, so it is natural to assume the larger model is the more capable one and the smaller one is the specialist. It is the other way round in kind. Clef is large because it carries a frozen multimodal backbone it needs in order to read screenshots and invoices; the trained part on top is a small schema head with rank-256 adapters. MathForm-8B is small because Lean 4 is a narrow target and the Qwen3-8B base was enough to hit it, with the real engineering sitting in the retrieval and verification pipeline that produced its training data rather than in its parameter count.

Size, in other words, tells you what each model had to carry, not how hard the problem it solves is. A 27B that reads a receipt and returns "billing, 0.98" and an 8B that emits compilable Lean are both doing exactly what they were built for, and the eight-billion-versus-twenty-seven-billion framing will lead you to the wrong one about half the time.

The decision, and the question neither vendor has answered

If you have a pipeline that ingests unbounded natural language and needs to route it before spending compute on it, the first move is a bounded classifier — Clef where its multimodal input matters and its numbers look good on your data, or Jev 1.13 where you want the same request shape at a lower list price and without tying your architecture to a vendor whose evaluation nobody has reproduced. The second move is a specialist generator, and if that generator is Lean 4, MathForm-8B is the open model built for exactly that, with a published pipeline and a corpus behind it.

What is missing on both sides is the same kind of evidence. Clef's Decision Index run is Cloudflare's own and has never been independently repeated; MathForm-8B's Pass@8 figures come from its own paper. Both are ambitious claims in an area where the honest test is cheap to describe and expensive to run — take data neither vendor trained on, apply the same pipeline, and publish the result. Until somebody does that for either model, the useful thing this comparison can tell you is a shape: one of these belongs at the front of your pipeline deciding what arrives, and the other belongs behind it doing the hard, narrow work, and neither is a substitute for the other at any parameter count.