MiniCPM-V 4.7 vs Gemma 4 12B: One Ships Benchmarks, the Other Ships a Picture of Them

Open MiniCPM-V 4.7's model card and scroll to the evaluation section. You will find three images: a performance-versus-inference-time frontier plot, an overall vision-language comparison, and an overall language comparison. You will not find a table. Now open Gemma 4 12B's card and scroll to the same place. You will find MMLU Pro, AIME 2026, LiveCodeBench v6, GPQA Diamond, Codeforces ELO, MMMU Pro, MATH-Vision and a document-parsing error rate, each with a number you can copy into a spreadsheet. Both models are open-weight, both are Apache 2.0, both are positioned as multimodal models that fit on hardware you own — and only one of them will let you check its work. The documented AI failures and agent incidents behind releases like this one are collected in the OrcaRouter AI Incident Archive where each entry carries its own sources, severity, confidence grade and disputed flag.

That asymmetry is the actual comparison. MiniCPM-V 4.7, released by OpenBMB on October 12, 2026, and Gemma 4 12B, released by G​oogle DeepMind on June 3, 2026, are not the same kind of object, and the difference is not size. It is how much of their own claim each vendor is willing to put in text.

What each vendor will tell you, in numbers

Gemma 4 12B's card publishes its scores against five siblings and a predecessor, with the evaluation harness named by benchmark. For the 12B Unified variant specifically, the vendor reports 77.2 on MMLU Pro, 77.5 on AIME 2026 without tools, 72.0 on LiveCodeBench v6, 78.8 on GPQA Diamond, a Codeforces ELO of 1659, 69.0 on Tau2 averaged over three runs, 83.4 on MMMLU, and — the number that most honestly frames what a 12B model is — 5.2 on Humanity's Last Exam without tools. Vision-side it reports 69.1 on MMMU Pro, 79.7 on MATH-Vision, and 0.164 average edit distance on OmniDocBench 1.5, where lower is better.

Those are vendor numbers. Nobody independent has to agree with them, and on a model this widely deployed some of them will not survive contact with a different harness. But they are falsifiable, which is the property that matters. If you get 62 on MMLU Pro from your own eval, you know something is wrong with your setup or with the card.

MiniCPM-V 4.7's card makes four numeric claims in prose and no benchmark table at all. The 35B-A3B variant decodes at 13 or more tokens per second using about 3 GB of memory on an iPhone 17 Pro Max. Lookahead Expert Scheduling delivers a 5× speedup on real devices. The 1B variant beats Qwen3.5 0.8B in overall score and inference efficiency. And in Instruct mode, 35B-A3B reaches a higher overall score in less inference time than Qwen3.6 35B-A3B. Everything else — every axis of the comparison — lives inside the plot images.

That is not dishonesty. Charts carry information, and a frontier plot is arguably a better summary of a speed-quality trade-off than a table of scores on different benchmarks measured at different batch sizes. But a chart cannot be quoted, cannot be re-derived, and cannot be compared against another vendor's chart without someone reading pixel positions. When the only public evaluation of a model is a picture, the correct posture is to treat the model as unmeasured until somebody measures it.

The two architectures are solving different problems

Read past the evaluation sections and the designs diverge sharply, which is the more useful signal than any single score.

Gemma 4 12B is encoder-free. There is no separate vision tower and no separate audio encoder: raw image patches and audio waveforms project straight into the language model's embedding space through lightweight linear layers. G​oogle's stated reason is latency and single-pass fine-tuning — one model, one training run, all modalities. The trade is that the company is betting a decoder-only transformer can learn to see well enough without a dedicated visual front end. Its own MMMU Pro figure of 69.1, against 76.9 for the 31B sibling that does carry a roughly 550M-parameter vision encoder, suggests the bet costs something in fine visual reasoning.

MiniCPM-V 4.7 goes the other way. It keeps a dedicated vision tower — 1152 hidden width, 27 layers, 980-pixel tiles — and spends its engineering on making that tower cheap. The intra-ViT early compression borrowed from LLaVA-UHD v4 cuts visual tokens inside the encoder rather than after it, and a mixed 4×/16× downsampling switch lets a caller choose between detail and cost per request. On the language side, the 35B-A3B is a 256-expert mixture activating 8 experts per token, layered into a 3:1 pattern of linear attention to full attention across 40 layers.

The practical consequence of that design is that MiniCPM's video path is unusually explicit. The card exposes max_num_frames at 128, a stack_frames parameter that composites sub-frames into a grid between main frames, and separate slice-count guidance for images (36) versus video (1). If you are building something that reads screen recordings or lecture video, that level of control over what gets sampled is the feature. Gemma 4 12B handles video too, as sequences of frames, but its card documents the format rather than the sampling economics.

The rows that actually decide it

• Parameters — 35.21B total, 8 of 256 experts active, plus a 1.55B dense sibling vs 11.95B dense

• Modalities in — text, image, video, with OCR and PDF parsing called out vs text, image, video and native audio

• Context — 262,144 tokens vs 262,144 tokens on the 12B, 128K on the E-series siblings

• Licence — Apache 2.0, weights downloadable vs Apache 2.0, weights downloadable

• Published evaluation — frontier and comparison charts, four prose claims vs a full numeric table across eight siblings

• Independent evaluation — no Artificial Analysis entry for either 4.7 variant as of this writing vs an Artificial Analysis page with intelligence placement and throughput

• Memory at BF16 — roughly 70 GB of weights for the 35B-A3B, about 3.1 GB for the 1B vs roughly 24 GB

• Quantised formats shipped — GGUF, bitsandbytes, AWQ, GPTQ, FP8, int4 vs the upstream checkpoint plus community GGUF and QAT builds from G​oogle and third parties

• Serving paths named by the vendor — SGLang, vLLM, llama.cpp, Ollama, Transformers, OpenVINO vs Transformers, with the wider ecosystem supplied by the community

• On-device story — iOS, Android, HarmonyOS apps plus Rockchip, Houmo, Intel, AMD and Arm day-0 support vs a design optimised for laptops and consumer GPUs

Three workloads, three answers

Screen and document understanding at volume. This is where MiniCPM-V 4.7's design is aimed and where its documentation is strongest. The 4× downsample mode, the 36-slice setting, the OCR and PDF-parsing cookbook entries, and the explicit frame-sampling controls for video all describe a pipeline for turning rendered content into structured text. Gemma 4 12B is credible here too — its OmniDocBench edit distance of 0.164 is a real document-parsing figure — but the vendor's card treats document parsing as one capability among many rather than the organising principle.

Anything that needs to hear. Gemma 4 12B takes audio natively, with CoVoST at 38.5 and FLEURS at 0.069 on its card. MiniCPM-V 4.7 does not. OpenBMB's audio work lives in the separate MiniCPM-o line, and the 4.7 card lists no audio modality at all. If your input includes speech, the comparison is over before it starts — one model needs a transcription step in front of it and the other does not.

A single consumer GPU with a fixed budget. Gemma 4 12B at 4-bit fits under 8 GB and leaves room for context; the 35B-A3B at 4-bit does not fit on one consumer card and its card's own performance story is about phones holding 3 GB resident, not about a GPU with 70 GB of expert weights to page. If the constraint is "one card you already own," the 12B is the answer and the 1B variant is the only MiniCPM that applies.

The gap a buyer should price in

Both models are free to download, so the licence line is a tie and the real cost is everything after it. For Gemma 4 12B, the ecosystem has had four and a half months to catch up: quantised builds from G​oogle itself, community GGUF conversions, QAT checkpoints, and independent indexing that lets you compare it to a hundred other models on the same scale. For MiniCPM-V 4.7, you are eight days into a new architecture — a new model_type, a new attention layout, a minimum Transformers version — and the vendor's own card opens with a note about CUDA version mismatches breaking video decoding through torchcodec.

That is not a criticism of the model. It is the standard tax on adopting anything in its first fortnight, and it is the reason a quiet release with a chart-only evaluation is a bad candidate for a production path and a good candidate for a side-by-side test. The way to make that test cheap is to keep the incumbent serving while you evaluate the newcomer, which is a routing problem rather than a modelling one.

OrcaRouter runs 200-plus hosted models behind one OpenAI-compatible endpoint at 0% markup, provider list price passed straight through, so the cost of trying a second model is a config change rather than a second contract. Its routing DSL can hold a primary and a fallback in a single call, and automatic failover keeps the primary path alive when a provider has a bad hour. The coverage note is the important part here: our catalogue carries Gemma 4 26B-A4B and Gemma 4 31B, not this 12B, and it does not carry MiniCPM-V 4.7 at all. Both of the models this article is about are checkpoints you run yourself — the router is what sits around them, not a way to call them.

What would settle it

The comparison is currently lopsided in a way that is easy to state and easy to fix. Gemma 4 12B has numbers you can argue with. MiniCPM-V 4.7 has claims you can only accept or reject: 13 tokens per second, 3 GB, 5× speedup, better than a named competitor on an unnamed overall score. Every one of those would be more useful as a table row, and OpenBMB is clearly capable of producing one — the family's earlier cards carried standard benchmark grids.

So the useful posture is asymmetric on purpose. If you need a decision this week, Gemma 4 12B is the model whose behaviour you can predict from published figures, and its audio support is a capability no amount of patience will add to MiniCPM-V 4.7. If you are building for phones, for HarmonyOS, for a Rockchip board, or for reading screens and documents where the 4× downsample path is exactly right, MiniCPM-V 4.7's engineering is more specific to your problem than anything G​oogle has shipped at this size — and the missing table is a reason to measure it yourself rather than a reason to dismiss it.

One last thing worth saying plainly, because it is the part that does not fit either vendor's framing. Neither of these models is competing for the same buyer as a frontier API. They are competing with the decision to not run a model locally at all. Gemma 4 12B's strongest argument is that four months of ecosystem means the boring path works; MiniCPM-V 4.7's strongest argument is that a 1B model at 3 GB runs somewhere a 12B cannot go. Pick the constraint first, then the model — the benchmark table, when it arrives, will only rearrange models inside a category you have already chosen.