LFM2.5-24B-A2B vs Gemma 4 12B: The Sparse Agent Against the Dense Multimodal One

Liquid AI's LFM2.5-24B-A2B and G​oogle DeepMind's G​emma 4 12B are the two open-weight checkpoints most likely to be sitting in the same download queue this month, and they are arrived at from opposite directions. LFM2.5-24B-A2B shipped on October 13, 2026 carrying 24.0 billion parameters of which only 2.45 billion are read per token, text in and text out, a 131,072-token context, and a benchmark table built entirely around tool use. G​emma 4 12B shipped on June 3, 2026 as an 11.95-billion-parameter dense model that reads text, images and audio and writes text, under Apache 2.0, with no revenue cap attached. One is a specialist agent engine that runs on the laptop you already own. The other is a generalist that sees and hears, runs on the same laptop in about 7 GB, and comes with a licence you never have to read twice.

So the question this page answers is not "which is better," because they were never built for the same job. It is narrower and more useful: if you are putting a private agent on a machine you control, which of these two do you download, and what do you give up either way?

The answer in one paragraph, for people who already know what they want

Take LFM2.5-24B-A2B if the agent does multi-step tool calling and nothing else — the vendor's own agentic numbers are far ahead of G​emma 4 12B's on the handful of rows comparable enough to line up, and the model always produces an explicit reasoning trace before it acts. Take G​emma 4 12B if the task involves a screenshot, a photographed document, a scanned page or an audio clip, if you want a reasoning trace switched off for latency, or if your company is anywhere near the $10M revenue line where Liquid's licence stops being free. Read on if you want the numbers behind those two sentences, because a fair amount of the comparison turns on how the two vendors measured.

Where the two checkpoints genuinely differ

• Parameter economics — LFM2.5-24B-A2B is a mixture-of-experts: 24.0B total, 2.45B active per token, 64 experts with top-4 routing across 40 layers (30 double-gated short-convolution blocks, 10 grouped-query attention). G​emma 4 12B is dense: 11.95B total across 48 layers, every parameter read on every token.

• What fits on disk — Liquid publishes a Q4_K_M GGUF at 14.5 GB and a QAT QAD-Q4_0 at 13.6 GB, with a 4-bit MLX build at 13.6 GB; 48 GB in BF16. G​oogle's QAT GGUF for the 12B is about 6.98 GB of weights plus a 0.18 GB multimodal projection file, and the Unsloth community builds run from roughly 4.2 GB at 2-bit to 7.12 GB at Q4_K_M.

• Context — 131,072 tokens for LFM2.5-24B-A2B, doubling the previous generation's 32,768. G​emma 4 12B carries 256K (262,144), with a 1,024-token sliding window and hybrid attention that interleaves local and global layers.

• Modality — LFM2.5-24B-A2B is text-only, and deliberately so. G​emma 4 12B takes text, image and audio, with no multimodal encoders at all: raw image patches and audio waveforms are projected straight into the same space as text tokens.

• Reasoning behaviour — LFM2.5-24B-A2B is reasoning-only in the vendor's phrasing: it always thinks inside a <think> block before answering. G​emma 4 12B's card describes an instruction-tuned model with no mandatory trace, which on a laptop is a latency win and a transparency loss.

• Licence — lfm1.0 is free for download, fine-tuning and commercial use up to $10M in annual revenue, then a commercial licence applies. G​emma 4 12B is Apache 2.0 with no revenue trigger; G​oogle's Terms of Use and Prohibited Use Policy sit alongside the weights as acceptable-use documents, not as licence restrictions.

• Vocabulary and training scale — LFM2.5-24B-A2B grew its vocabulary from 65K to 128K by in-place tokenizer expansion and scaled pretraining from 17T to 33T tokens, with 300B of mid-training tokens at full context and an agentic RL stage inside real harnesses. G​emma 4 12B carries a 262K vocabulary shared across the family and was pretrained on 140+ languages with claimed out-of-the-box support for 35+.

The one benchmark both vendors ran

Comparing two different launch tables is normally a trap. Here there is exactly one row where the two companies measured the same thing on comparable terms, and it is worth isolating precisely because it is so lonely.

GPQA Diamond — Liquid reports 74.42 for LFM2.5-24B-A2B; G​oogle's G​emma 4 family table reports 78.8 for the instruction-tuned G​emma 4 12B. Both are vendor-reported and unreproduced. The dense 12B edges ahead on graduate-level science questions, which matches Liquid's own framing that its model is not the one to ask knowledge questions of — the launch post advises pairing it with retrieval or web search rather than trusting its weights.

Everything else has to be read within its own table, and the honest summary is that the two tables measure nearly disjoint things. Liquid's sixteen-benchmark table covers AutomationBench (27.69), BFCLv4 (65.01), ToolSandbox (80.71), LiveClaw (48.09), PinchBench (83.18), τ³-Bench Banking (8.04), WebArena Verified (26.75), IFEval (96.75), IFBench (74.69), Multi-IF (84.38), MATH-500 (96.04), SWE-Bench Verified via OpenCode (51.00), SciCode (32.71), AA-Omniscience (−24.90) and HLE (7.40). G​oogle's family table covers MMLU Pro (77.2), AIME 2026 without tools (77.5), LiveCodeBench v6 (72.0), a Codeforces ELO of 1659, MMMU Pro (69.1) and MRCR v2 at 128K with eight needles (43.4).

Only one overlap, and no shared harness. Anyone publishing a combined scoreboard from those two tables is inventing a comparison that neither company ran.

Independently, and this is the sharper edge

Artificial Analysis has scored G​emma 4 12B: an Intelligence Index of 14, ranked 20th of 141 models in its comparison class, at a measured 110.9 tokens per second and a blended third-party cost of $0.10 per million input and $0.30 per million output tokens. Those are independent numbers on a model that has been public for four months and has an ecosystem behind it.

LFM2.5-24B-A2B has nothing equivalent. It is a day old, no third party has published a score, and Liquid's own edge figures — 184 tokens per second on an Apple M5 Max, 114 on an AMD Ryzen AI Max+ 395, "up to 1.7× faster than comparable MoEs on-device" — carry no baseline and no reproduction. For a vendor-measured model against an independently measured one, the asymmetry is the finding: one of these two numbers you can check yourself today, and it is not the newer one.

If you would rather not bet a production path on a four-day-old vendor table, that is exactly what automatic failover is for. Point the agent at LFM2.5-24B-A2B, keep a scored model behind it, and let the router move the request when the harness starts failing — no second contract, no code change beyond the fallback list.

Cost, in the two currencies these run in

Both are local-first models, so the real bill is hardware and electricity. On the hosted side, Liquid prices API access at $0.10 per million input tokens ($0.05 cached) and $1.00 per million output through its own Console, with $2 of starting credit. G​oogle publishes no per-token price for G​emma 4 12B at all — the $0.10/$0.30 blend above is a third-party median across platforms serving the open weights, not a list price, and treating it as one would be wrong.

Here is the part that matters for a team evaluating both: neither checkpoint is on our catalogue today. We do serve the reference points this decision keeps circling back to — qwen/qwen3.6-35b-a3b, the model Liquid measured against with half again the parameters, plus google/gemma-4-26b-a4b-it, obsidian/gemma-4-26B-A4B and google/gemma-4-31b-it from the same G​emma 4 family — all behind one key, all at 0% markup on the provider's own list price, so a vendor price change reaches you the day it changes rather than the day someone re-signs a contract. Run your own task against the hosted G​emma 4 sizes, then decide whether the 12B on your laptop is redundant.

What I would actually do

Download both quants. G​emma 4 12B at Q4_K_M is around 7 GB and will tell you in an hour whether your task needs eyes and ears — if it does, the decision is over and LFM2.5-24B-A2B is not in the running at all, because it cannot see. If it does not, load the LFM2.5-24B-A2B QAD-Q4_0 at 13.6 GB and run your real harness against it, since the entire argument for that checkpoint is multi-step tool calling and no static benchmark will settle it for your tools.

Two things to keep in mind while you do. First, both vendors' tables are unreproduced, and the only independently scored model in this comparison is the older, denser one — which is a reason to trust the G​emma 4 12B numbers more, not a reason to assume the newer model is weaker. Second, the licence gap is real and asymmetric in a way that rarely shows up in benchmark posts: G​emma 4 12B is Apache 2.0 forever, while LFM2.5-24B-A2B becomes a commercial conversation the moment your company crosses $10M in revenue. For an individual builder that is invisible. For a product team shipping an on-device agent, it is a line item in the next planning cycle, and it is the one dimension where the smaller, older, multimodal model is unambiguously the safer default.

Comparisons like this one are also where the failure modes show up: the documented AI incidents and agent failures behind models in this class are collected in the OrcaRouter AI Incident Archive , where each entry carries its own sources, severity, confidence grade and disputed flag.