Solar Mini 4 vs MathForm 8B: One Model Answers the Question, the Other Makes It Checkable

Put Solar Mini 4 next to MathForm 8B and you are comparing two models that do not compete for the same token. Solar Mini 4 is Upstage's 35-billion-parameter sparse mixture-of-experts, released 22 September 2026 with roughly 3 billion parameters active per token, and it answers questions: read a long document, reason over it, return a written result. MathForm 8B is OpenBMB's 8-billion-parameter autoformalizer, published 14 August 2026 under Apache 2.0, and it does something no reasoning model does at all — it takes a natural-language mathematical statement and rewrites it as a Lean 4 theorem that a proof assistant's compiler will type-check. One produces answers. The other produces machine-checkable questions, and the second is the rarer commodity.

The comparison is worth making anyway because both of them end up in the same kind of pipeline, and because the two have opposite strengths in a way that makes the choice unusually clean. Upstage's model is strong on long documents, weak on hard reasoning, and expensive per completed task. OpenBMB's model is narrow to the point of being useless outside its niche, has never been scored by an independent evaluator at all, and costs nothing to run once you have a GPU.

Side by side, on the things that differ

• What it is — Solar Mini 4 is a general reasoning model with a 1M-token context (512K by Upstage's own documentation) and a measured 24.1 on the Artificial Analysis Intelligence Index. MathForm 8B is a single-task autoformalizer with no published index score, no independent evaluation, and no general capability claims.

• Parameters — 35B total / 3B active, sparse, for Solar Mini 4; 8.19B dense for MathForm 8B, fine-tuned from Qwen3-8B on OpenBMB's FormalVerse corpus of roughly 367,000 verified Lean 4 examples.

• Licence and delivery — Solar Mini 4 is proprietary, reached through Upstage's API and third-party platforms. MathForm 8B is Apache-2.0 weights with a chat template, four safetensors shards, and a Transformers, vLLM or SGLang deployment path behind an OpenAI-compatible endpoint.

• Price — Solar Mini 4 at $0.10 / $0.01 cached / $0.40 per million tokens, currently 50% off through 22 October 2026. MathForm 8B has no rate card; its cost is the GPU you put under it.

• Speed — 204 output tokens per second for Solar Mini 4 on Artificial Analysis's harness, with 88,300 output tokens per index task and about 430 seconds of work per task. MathForm 8B's output is one Lean 4 theorem header, a few hundred tokens at most.

• Independence of its numbers — Solar Mini 4 has been run by a third party. MathForm 8B has not: every figure in its paper is the authors' own, and the model is too new and too specialised to have attracted an independent run.

Where the two models meet, and where they do not

The failure modes are what make this pairing interesting, because they are exact opposites. Solar Mini 4's weakest published result is Terminal-Bench 4.0 at 1%, with AutomationBench-AA at 22.3% — it is bad at verifying its own work in an environment that gives it feedback. MathForm 8B's entire design premise is verification: OpenBMB distinguishes a Syntax Check, which asks whether the Lean 4 statement compiles, from a Consistency Check, which asks whether the statement it compiled actually means the same thing as the informal problem. The classic failure it exists to prevent is a statement that type-checks while silently weakening the claim.

That is a real distinction and it is worth being precise about. A reasoning model producing a proof has a well-known self-verification problem: nothing in the generation tells you whether the answer is right. A compiler does. If your workflow is "convert a problem bank into something a machine can check," MathForm 8B is doing the half of the job that Solar Mini 4 cannot do at all, and the leftovers are not a matter of degree.

The vendor numbers, labelled as such: OpenBMB's paper reports an average Pass@8 of 88.06% under Syntax Check and 72.37% under Consistency Check across six benchmarks, and claims those beat several specialised 32B autoformalizers. None of that has been reproduced by a third party. The 16-point drop between the two checks is the more informative number — it measures how often a compiled statement is not quite the problem you asked about, and it is the reason Consistency Check exists as a separate column.

Why a team would run both

Because the natural pipeline has a cheap, high-volume stage and an expensive, low-volume stage. MathForm 8B runs on your own hardware and converts informal problems into Lean 4 headers, with a compiler available to reject bad output before a human sees it. Solar Mini 4 handles what happens around that: reading the supporting paper, extracting the assumptions, summarising the result in Korean or English, and routing the work. The two never compete for the same request, and the smaller one is the one that owns the part of the job with an objective pass/fail signal.

Neither model is something we can route to you — Upstage's models are not on OrcaRouter and neither is OpenBMB's — so if you want Solar Mini 4 it comes from Upstage's own API, and if you want MathForm 8B it comes from Hugging Face and your own GPU. What we can do is put the rest of that pipeline behind one key: more than 200 models at 0% markup over provider list price, automatic failover across providers, and a routing DSL that lets you chain models into a single call. In a workflow where a small specialist does the formalisation step and a large generalist does everything around it, being able to change the generalist by editing a model string — instead of shipping a new integration — is the difference between a two-week change and an afternoon.

What to take away

If your question is "which of these should I use," the honest answer is that it depends on whether you need an answer or a formalisation, and no benchmark will settle it. If your question is "is MathForm 8B worth the download," the answer is yes for one narrow job and no for anything else — it is a formalizer, not a prover, and the proof obligation stays open in the output it produces. If your question is "is Solar Mini 4 worth $0.36 a task," the answer is yes for long documents at volume and no for anything that requires it to check its own work.

Sourcing, to close. Every Solar Mini 4 figure above is an independent measurement by Artificial Analysis except the context window, which is Upstage's own figure and disagrees with theirs. Every MathForm 8B figure is vendor-reported from arXiv 2608.14221 and unreproduced, and its release was unannounced — the weights, the FormalVerse dataset and the paper all appeared on 14 August 2026, with no vendor blog post and no independent benchmark run to date.