LFM3-1.7B-Base vs LFM2.5-2.6B-Base: Fewer Parameters, Fewer Tokens, More Context

The arithmetic goes the wrong way. LFM3-1.7B-Base, published on October 26, 2026, is 36% smaller than LFM2.5-2.6B-Base, which Liquid AI released on August 4, 2026 — 1.73 billion parameters against 2.69 billion. It was also trained on fewer tokens: 22 trillion against roughly 34 trillion. By the two numbers the industry usually uses to rank pre-trained models, the newer checkpoint should be worse. Liquid's claim is the opposite, at least for the post-trained model built on top of it, and the interesting question for anyone choosing a trunk to fine-tune is which of those differences is real and which is a story told with a benchmark suite. Running the comparison should cost compute, not a second contract. 0% markup OrcaRouter passes provider list prices through at 0% markup.

This is a comparison inside one family, which makes it unusually clean. Same vendor, same license family, same 128,000-token vocabulary, same hidden size of 2,048, same base-to-base role. The two checkpoints differ in architecture, data volume, context length and the amount of post-training baked into the trunk. Almost everything else is held constant.

The delta, dimension by dimension

• Parameters — 1.73B for LFM3-1.7B-Base against 2.69B for LFM2.5-2.6B-Base. A 35.7% reduction, which is where the "35% fewer parameters" line in Liquid's launch post comes from.

• Layers — 24 against 30. The newer model is shallower as well as narrower.

• Layer mix — thirteen gated short convolutions, five sliding-window attention blocks and six full-attention blocks for LFM3; twenty-two convolutions and eight full-attention blocks for LFM2.5, with no sliding-window tier.

• Attention — sixteen query heads and four key-value heads in LFM3, against thirty-two and eight in LFM2.5. Both keep head dimension 128 and hidden size 2,048.

• Feed-forward width — 7,680 for LFM3 against 10,752 for LFM2.5. This is where most of the parameter reduction actually lives.

• Convolution cache — seven for LFM3 against three for LFM2.5, with a 256-token sliding window added.

• Context — 262,144 tokens against 128,000. The newer checkpoint doubles it.

• Pre-training tokens — 22 trillion against roughly 34 trillion, with the last 3.5 trillion of the newer run devoted to curated mid-training data including agentic traces and tool calls.

• Artifacts — LFM3-1.7B-Base ships as BF16 safetensors only, at 3.46 GB. LFM2.5-2.6B-Base ships in Safetensors plus GGUF, ONNX and MLX builds, runs in under 2.5 GB quantized, and is supported by Transformers, vLLM, SGLang, llama.cpp and MLX.

Where the parameters went

The reduction is not spread evenly, and that matters for anyone estimating fine-tuning cost. Embeddings are tied in both models and the vocabulary is identical at 128,000 tokens, so the embedding matrix is unchanged. Hidden size is unchanged at 2,048. The savings come from three places: six fewer layers, half as many attention heads, and a feed-forward block roughly 29% narrower.

Narrowing the feed-forward block is the cheapest way to shrink a transformer, because that block holds most of the parameters in each layer. Halving the attention heads reduces the key-value cache as well as the weights, which is why a model with a doubled context can still be described as fitting in about 1 GB at 4-bit. The trade is capacity: a narrower MLP has less room to store factual associations, and a smaller head count gives the model fewer independent attention patterns to work with. Liquid's argument is that better data and a better layer mix more than compensate. That argument is testable, and as of publication nobody outside the company has tested it.

Where the context came from

Doubling context from 128K to 256K without doubling the attention bill requires not attending globally. LFM3's stack does that by sending five of its twenty-four layers to a 256-token sliding window and thirteen to convolutions, leaving only six layers that see the entire sequence. The config also carries a partial rotary factor of 0.5 on the global-attention layers, applying rotary embeddings to half of each head's dimensions, and a rotary theta of one million on those layers against ten thousand on the sliding ones.

Liquid's launch post gives one concrete number for what this buys: at 128K tokens, the text model's KV cache is 1.6 GB. For LFM2.5-2.6B-Base, no equivalent figure is published. The comparison is therefore directional rather than exact — the newer architecture is designed to hold a longer context in less memory, and the vendor's own figure is the only measurement available.

Why fewer tokens is the interesting claim

Thirty-four trillion tokens down to twenty-two trillion is a 35% reduction in pre-training compute at the data level, and it is the opposite direction from most model generations. The usual story is more data, more parameters, better model. Liquid's story is fewer of both, with a new architecture and a data mix that adds scientific papers, repository-level code, mathematics and a wider spread of knowledge sources.

The 3.5-trillion-token mid-training stage is the part that distinguishes this checkpoint from a straightforward pre-train. It uses curated, higher-quality data with a larger share of agentic traces and tool calls, and it is where the context window is extended. That stage is also why the base model carries a chat template: the mid-training data was formatted as conversations, with and without thinking traces.

None of this can be verified from outside. Token counts are self-reported, data mixes are not published, and there is no ablation showing which change produced which gain. The claim is plausible — data quality has repeatedly beaten raw volume in the last two years — but plausible is not measured.

What a base-to-base comparison can settle

It can settle specifications, cost and compatibility, and this page does. It cannot settle quality, because neither checkpoint has a published benchmark. Liquid published no evaluation for LFM2.5-2.6B-Base and has published none for LFM3-1.7B-Base either. Both cards recommend the checkpoint for extensive fine-tuning and point readers elsewhere for chat.

The only quality evidence in the family is the post-trained pair. Liquid reports LFM3-1.7B at an average of 49.37 on its own suite against 31.55 for LFM2.5-2.6B, with the newer model using 35% fewer parameters. That is a large gap, and it is a vendor figure measured on a vendor-chosen suite, comparing two models that differ in post-training as well as in trunk. It is evidence that the LFM3 recipe produces a better instruct model. It is not evidence that the base checkpoint is a better starting point, because the post-training on top of it could account for the whole difference.

Migration notes for anyone on LFM2.5

The two checkpoints share a vocabulary size, a hidden size and a tokenizer family, which makes migration look easier than it is. They do not share modeling code: LFM2.5 loads as Lfm2ForCausalLM and LFM3 as Lfm3ForCausalLM, both through custom code with trust_remote_code=True. A training script written for one will not load the other without changing the model class, and the chat templates are different in their handling of thinking traces.

The bigger operational difference is packaging. LFM2.5-2.6B-Base has GGUF, ONNX and MLX builds and runs under llama.cpp, vLLM, SGLang, Transformers and MLX. LFM3-1.7B-Base ships as a single BF16 safetensors file; the quantized and cross-platform builds exist for the post-trained LFM3-1.7B, not for the base checkpoint. If your pipeline depends on a GGUF artifact, you are converting it yourself and validating the result.

Fine-tuning support is the counterweight. LFM3-1.7B-Base links TRL notebooks for supervised fine-tuning and continued pre-training, and the docs cover the path from there. The smaller checkpoint, the 256K context and the agentic mid-training are all aimed at exactly the kind of domain adaptation those notebooks demonstrate.

Which trunk to start from

Start from LFM3-1.7B-Base if you are beginning a new fine-tune, your task benefits from long context, and you can work in BF16 with Transformers or convert the weights yourself. The 35% parameter reduction is real money on a single-GPU training run, the doubled context is a genuine capability change, and the chat template shortens the supervised phase.

Stay on LFM2.5-2.6B-Base if you already have a working pipeline, depend on a GGUF or ONNX artifact, or need a checkpoint with a year of community tooling behind it. The newer model is three days old. Its instruct sibling has quantized builds and day-one support in vLLM, SGLang, llama.cpp, Ollama, LM Studio, MLX and ONNX Runtime, but the base checkpoint does not, and that gap is the practical cost of being early.

If you are unsure, run the experiment rather than reading the marketing. Take a small slice of your own data, run the same continued-pre-training recipe against both trunks for the same number of steps, and evaluate on a held-out set you built before you started. That is a few GPU-hours and it answers a question no vendor table can. When the two fine-tunes exist, put both behind one endpoint: OrcaRouter's routing DSL can split traffic between them, and automatic failover means a bad experiment does not become a bad production day. The platform passes provider list prices through at 0% markup, so the cost of running the test is the cost of the compute, not a second contract.

The number to watch

The claim that would settle this is not another average. It is a base-model evaluation — any base-model evaluation — published by Liquid or by a third party, comparing LFM3-1.7B-Base and LFM2.5-2.6B-Base after an identical fine-tuning recipe. Until that exists, the generational delta is documented in the architecture and the config, plausible in the token counts, and unproven where it matters. The newer checkpoint is smaller, longer-context and better packaged for training. Whether it is better is a question the repository does not answer.