ARTEMIS vs LFM2.5-2.6B-Base: the checkpoint that cannot do the job, and the harness that needs it to

Four items sit on Google's ARTEMIS roadmap, and exactly one of them requires a model that does not obviously exist yet: on-device lightweight vision-language models, for low-latency, privacy-first automation. The obvious candidate for a job like that is something in the size class of LFM2.5-2.6B-Base — Liquid AI's 2.69-billion-parameter pre-trained checkpoint, published as open weights with a 131,072-token context and a footprint small enough for a phone. It is also, as shipped, incapable of filling that slot, and the reasons are worth understanding before anyone puts the two in a comparison table. Google's ARTEMIS is a natural-language Android automation harness, open-sourced under Apache 2.0 in August 2026, that drives a real handset through ADB and claims a 99%+ completion rate on Google Research's AndroidWorld benchmark. LFM2.5-2.6B-Base is raw pre-trained material with no instruction tuning, no chat template, and — deliberately — no published benchmarks, intended for teams that will post-train it themselves. One is finished software with an unfinished model problem. The other is unfinished weights with a finished licence problem. Neither is a substitute for the other, and the place where they genuinely meet is not the place you would guess.

What LFM2.5-2.6B-Base actually is

Strip the branding and it is a carefully built foundation for on-device work, with specifications that read like someone optimised for memory bandwidth rather than leaderboard position.

• Size — 2.69B parameters at bfloat16 in a single ~5.39 GB shard, marketed as "2.6B".

• Architecture — 30 layers in a hybrid split: 22 double-gated short-convolution blocks over 8 grouped-query attention layers, hidden width 2048, 32 attention heads against 8 KV heads, tied embeddings. Same code>Lfm2ForCausalLM/code> class as the previous generation, so no custom modelling code is required.

• Training — roughly 34 trillion tokens, with a dedicated mid-training phase for context extension.

• Vocabulary — 128,000 tokens, doubled this generation to handle non-Latin scripts better. About 262M parameters, roughly a tenth of the model, sit in the tied embedding.

• Context — the model card and config disagree here, and it is worth knowing: the documentation advertises 131,072 tokens while code>config.json/code> sets code>max_position_embeddings/code> to 128,000. Budget for 128K and treat anything above it as unverified.

• Languages — sixteen: English, Arabic, Chinese, French, German, Hindi, Indonesian, Italian, Japanese, Korean, Polish, Portuguese, Russian, Spanish, Thai, Vietnamese.

• Benchmarks — none. Liquid publishes no evaluation for the base checkpoint, and the model card frames the omission as intentional: this artefact exists to be post-trained, not to be measured in its raw state.

That last line is the whole point of the release, and it is also the reason a "X vs LFM2.5-2.6B-Base" comparison has to be handled carefully. A base checkpoint has no opinion about anything. Ask it to plan an Android workflow and it will continue your text probabilistically, because it has never been taught to answer. The card recommends it only for cases that require heavy fine-tuning: a language-specific assistant, a domain-specific one in a regulated vertical, training on proprietary data, or as a distillation student. For anything out of the box, including tool calling, Liquid points you at the post-trained LFM2.5-2.6B instead.

What ARTEMIS needs from a model, which is not what a base checkpoint offers

ARTEMIS is a control loop with two modes. Flash is a reactive observe-and-act cycle at roughly 3–5 seconds per step. Pro is a multi-agent graph at roughly 15–40 seconds per step, with a Planner holding a living Markdown plan, an Operator with the full toolset, and a read-only Checker verifying checkpoints at four levels from code>off/code> to code>strict/code>. Both modes ask the same thing of the model underneath: look at a screenshot, pick one action from a fixed toolset, and do it again in a second or two, a hundred times, without losing the thread.

That is a demanding ask — instruction following under a rigid schema, grounded visual understanding, and long-horizon coherence — and it is exactly the set of skills a base checkpoint has not been given. ARTEMIS's own tested backends are hosted models: Gemini, Claude, GPT-4o, Qwen-VL. Every one of them is instruction-tuned for tool use, and every one of them is large.

So the gap is not one of size. A 2.6B post-trained model can hold a tool-calling loop — Liquid's own post-trained sibling claims to outperform Gemma 4 E2B-it and E4B-it across all of its instruction-following evaluations and nearly all of its tool-use ones, though those are vendor-reported numbers with no independent verification. The gap is that a base checkpoint has had none of that training applied, so it cannot be dropped into a harness at all.

The licence is the sharper difference

This is where the matchup actually gets decided, and it is the part most comparisons skip.

• ARTEMIS — Apache 2.0. Use it commercially, fork it, ship it inside a product, no revenue condition. The one obligation is preserving notices and stating your changes, an obligation that the project itself had to repair publicly in September 2026 after Minitap alleged that 228 of ARTEMIS's 229 files matched its own Apache-2.0 code>mobile-use/code> project and that author names had been stripped by force-push. The repository now carries a Minitap credit line.

• LFM2.5-2.6B-Base — the LFM Open License, a custom licence rather than Apache or MIT. Below $10M in annual revenue the grant is broad, perpetual and royalty-free. At or above that threshold, the licence does not extend to commercial use at all, and you must contact Liquid. The important detail for anyone building on it: derivative works inherit the same terms. The checkpoint you post-train is not a new thing you own outright — it carries the revenue-conditioned licence forward, with a carve-out for qualified non-profits.

Put those two facts next to each other and the decision inverts depending on who you are. A funded company that wants to ship a phone-side agent has an Apache-2.0 harness it can use freely and a checkpoint it may not be able to use commercially at all. An individual developer or a startup under the threshold has both, and the licence is a footnote. Neither vendor is being unreasonable — Liquid is a company protecting its commercial tier, Google is open-sourcing test tooling — but "open weights" and "open weights" are not the same permission, and a comparison that stops at parameter counts will not tell you which one you have.

The family, because the base checkpoint is the wrong door into it

If the goal is an on-device Android agent, three siblings matter more than the base, and the one the ARTEMIS roadmap actually needs is not the obvious one.

• LFM2.5-2.6B — the post-trained agentic sibling. Vendor-reported throughput is around 30 tokens per second on phone-class hardware, 113 on a Ryzen AI Max+ 395 and 220 on an Apple M5 Max, running in under 2.5 GB.

• LFM2.5-VL-3B — the vision-language edge model, built on the same base with a SigLIP2 400M NaFlex encoder, with vendor-reported grounding precision@1 lifted from 57.1 to 87.9 on RefCOCO and, by Liquid's account, performance on on-screen UI elements that beats much larger Gemma models and comes within 0.7% of a 4.7B Qwen 3.5. Unverified, but it is the only member of this family that can see a screen.

• LFM2.5-230M — the extraction and classification tier, explicitly not recommended for reasoning-heavy work.

Which is the uncomfortable conclusion for the base checkpoint in this matchup: ARTEMIS's roadmap item is a vision requirement, the base is text-only, and the family member that fills the slot is the VL variant that has already had both the vision encoder and the post-training applied. The base checkpoint's role in an Android automation stack is not to drive the phone. It is to be the raw material underneath whichever small model eventually does.

Where they do meet: the flywheel nobody has built yet

Here is the one connection that is real rather than rhetorical, and it runs backwards from the usual direction. ARTEMIS's most underrated feature is not the agent — it is the exhaust. Every run captures crash stacks, keyframe screenshots, a session ledger of compressed steps and a diagnostic report, and Pro opens an "execution incident" whenever an action fails and keeps it in context until a later success resolves it. That is a labelled corpus of exactly the moments where a UI agent's perception was wrong.

Pair that with a checkpoint whose entire purpose is post-training and you have an obvious loop: run the harness on a hosted model, collect the traces where it stumbled on your app, and fine-tune a 2.6B model on precisely those frames. It is the kind of proprietary dataset Liquid's own card cites as the justification for releasing a base checkpoint at all — "training on your own data" — and the revenue-gated licence means it is a route that makes sense for teams under the threshold and requires a conversation for teams above it.

To be explicit about the status of that idea: nobody has published this loop, neither vendor suggests it, and there is no evidence either side has tested it. It is a proposal, not a result, and it should be read as one. But it is the only framing in which these two artefacts are collaborators rather than a category error.

If you get as far as fine-tuning, the comparison set is the other half of the problem. LFM2.5-2.6B-Base is not on our catalogue — no LFM2.5 variant is — so that checkpoint comes from Liquid's own distribution and the usual third-party hosts. Where a routed catalogue earns its place is benchmarking your fine-tune against the models it has to beat on tool use, on one key and one bill, with failover so a provider's bad afternoon does not become your evaluation's bad afternoon. That is a genuinely useful thing to have when the whole point of the exercise is a head-to-head you intend to act on.

So which one is your problem

If you have an Android app and you want it tested automatically this quarter, you want ARTEMIS, and LFM2.5-2.6B-Base is not part of the answer — you will run ARTEMIS against a hosted vision model, pay per step, and the roadmap item about on-device VLMs will arrive eventually and solve a cost problem you have not measured yet.

If you are building a product that must run on a handset with no network, you want the small-model route, and LFM2.5-2.6B-Base is the beginning of that project rather than any part of its solution: you will post-train it, you will read the LFM Open License against your revenue projection before you write the first training script, and if your agent needs to see a screen you will end up on the VL variant instead.

The one thing not to do is put these two side by side, declare the harness more capable, and move on. The useful comparison is between the two complete stacks — hosted model plus harness, versus fine-tuned small model plus a framework you build — and only one of those has a published success rate on a public benchmark, which is worth exactly as much as the fact that the other one has no cloud bill at all.

Where a routed catalogue earns its place is benchmarking your fine-tune against the models it has to beat on tool use, on one key and one bill, with failover so a provider's bad afternoon does not become your evaluation's bad afternoon.

LFM2.5-2.6B-Base is not on our catalogue — no LFM2.5 variant is — so that checkpoint comes from Liquid's own distribution and the usual third-party hosts.