Gemini 4 Carbon Review: Fourth on the Index, Slow to Answer

Gemini 4 Carbon sits fourth on the independent board and is priced like a mid-tier model, which is an unusual combination to review honestly: the Gemini 4 Carbon API page quotes $2.00 per million input tokens and $10.00 per million output, while the score it posts is 60 on Artificial Analysis Intelligence Index v5.0.1, fourth of 78 models. The question a review has to answer is not whether 60 is good. It is what you pay for in wall clock and in failure rate to get it.

What it is

A multimodal model in Google's Gemini 4 family with a 1,048,576-token context window and a 65K output ceiling. It takes text, images, audio, video and documents and returns text, with native tool use and structured JSON output. It answers on both the OpenAI-compatible endpoint and the native Gemini generateContent API, and Zero Data Retention is available.

The parameter set tells you what kind of model it wants to be: reasoning, include_reasoning, structured_outputs, tools, tool_choice, seed, temperature, top_p. There is a seed and a response format on the same list, which is the signature of a model positioned for extraction and agentic pipelines rather than for creative writing.

The score, and the version it was scored on

The 60 is not comparable with older numbers you will find quoted for other models, and this is the single most common error in comparisons published this month. Intelligence Index v5.0.1 is a different evaluation set from v4.3 and v4.3.2, so a v4.3 score and a v5.0.1 score do not belong in the same table. Within the v5.0.1 cohort, Carbon's 60 puts it fourth of 78, behind Claude Fable 5.2 at 65 and ahead of everything else that has been re-scored.

The nine evaluations behind that number are AA-Briefcase v1.1, AutomationBench-AA, Terminal-Bench 4.0, FrontierCode-AA 2.0, AA-Omniscience v1.1, GDP.pdf, HLE-Diamond, CritPt v2 Hard Subset and Terminal-Bench-Science 0.1. That is an agentic and scientific mix — terminal work, automation, document grounding, hard reasoning — not a coding-leaderboard mix. A model can score well here and still lose on a repository-scale refactor, so read the breakdown before you conclude anything about your own use case.

The part the headline number hides

Carbon's time to first token in the AA harness is 64.7 seconds, rank 69 of 77 models with latency data. Total wall clock per task is 28.4 minutes across the nine evaluations. Output speed once it starts writing is 72 tokens per second, which is unremarkable but not the problem; the problem is the think time before the first token.

Running the index against Carbon cost $5.48 in provider spend, which at a blended $2.70 per million tokens is a lot of tokens for one benchmark pass. AA also reports verbosity of 2.69 percent alongside 205 million output tokens for the run. Do not read the percentage as a concision score: Claude Fable 5.2 posts a far lower 0.97 percent with a higher 275 million output tokens, so the two figures move independently.

If the price is the reason you are reading this, the LLM API pricing comparison puts a blended $2.70 per million next to the rest of the catalogue, including the models that undercut it and the ones that cost several times more for a comparable score.

Through the gateway, the numbers look different

AA's 64.7 seconds is a benchmark harness letting the model think on hard problems. On OrcaRouter's live traffic over the last seven days, the p50 time to first token for Carbon was 6.89 seconds and the p95 was 10.00 seconds, with 226 tokens per second of output over 9.6 million tokens served. Both are true. A chat product sees something near the first number; a batch evaluation clone sees something near the second.

The figure that deserves more attention than either is the error rate: 39.7 percent of requests returned non-2xx across all upstream providers in that seven-day window, against 10.8 percent for Claude Fable 5.2. Whatever the cause — capacity, rate limits, provider rotation — a product that calls Carbon without a retry budget and a fallback model will show users failures at a rate no benchmark chart predicts. That is a routing question, and it is the one thing to test before the second week.

Where it belongs in the catalogue

Against its own siblings, Carbon is the multimodal member of the Gemini 4 line rather than the cheap one: input price is identical to Gemini 3.1 Pro at $2.00, output is $2.00 cheaper, and the difference that matters is that Carbon takes audio and video where 3.1 Pro takes text and images. If your pipeline never touches a video file, you are paying for a capability you will not use.

Against the rest of the field the decision usually comes down to two questions. If it is a Google-internal choice, Gemini 4 Carbon vs Gemini 3.1 Pro is the comparison that matters. If it is cross-vendor, GPT-6.1 Sol vs Gemini 4 Carbon and Claude Opus 5.5 vs Gemini 4 Carbon cover the two most common alternatives, and the LLM models list has the full catalogue if neither is the one you were considering.

Verdict

Carbon earns its fourth place on capability per dollar of input, and it earns it in a way that suits long, multimodal, structured work rather than conversation. The costs that do not appear in the price table are the two that decide whether it works for you: 64.7 seconds of thinking before the first token under a benchmark harness, and an error rate near 40 percent on live traffic. Route around both — a fallback chain, a retry budget, and an async queue for anything that is not interactive — and the price-to-score ratio is genuinely hard to beat. Put it behind a synchronous UI with no fallback and you will conclude the model is worse than it is.