SOV-Transcribe-1 vs Gemini 3.5 Transcribe: Three-Tenths of a Point, and Everything Else

On the Artificial Analysis speech board the two models sit 0.32 percentage points apart — SOV-Transcribe-1 at 2.28% AA-WER, Gemini 3.5 Transcribe at 2.60% — and that is the smallest gap in this entire comparison. Everything else is wider. Gemini 3.5 Transcribe covers 85-plus languages against seven, identifies up to eight speakers where the Salesforce model identifies none, emits word-level timestamps, and has a separate live endpoint for streaming. SOV-Transcribe-1 is open weights under Apache 2.0, runs on a GPU you own, and accepts a keyword list that measurably fixes names and tickers. Choosing between them is not a question about accuracy. It is a question about whether you are buying a transcript or buying a pipeline.

What each one is

SOV-Transcribe-1 is Salesforce's first open-weight speech-augmented language model, published on 12 November 2026 as salesforce-sovereign-intelligence/SOV-Transcribe-1: a Whisper large-v3 encoder feeding a Gemma 4 26B A4B mixture-of-experts decoder, 26.6B parameters total and 4.6B active, in a 53 GB BF16 checkpoint. It is non-streaming, handles up to 30 minutes of 16 kHz audio per request, and is controlled by instructions in the user message rather than by parameters.

Gemini 3.5 Transcribe is Google's hosted speech-to-text model, last updated in August 2026 and reachable through the Gemini API as gemini-3.5-transcribe, with a WebSocket streaming sibling called gemini-3.5-transcribe-live. It takes audio up to an hour per request — thirty minutes when diarization or timestamps are switched on — and returns text plus word annotations. There are no weights to download and no server to size.

The accuracy gap, decomposed

AA-WER v2 is a weighted composite: AA-AgentTalk at 50%, VoxPopuli-Cleaned at 25%, Earnings22-Cleaned at 25%. Splitting the two models across those components is more informative than the headline, because the Salesforce model's advantage is not evenly spread.

• AA-AgentTalk — SOV-Transcribe-1 at 1.79%. This is the half-weighted synthetic conversational set, and it is where the Salesforce model is strongest relative to anything else on the board.

• VoxPopuli-Cleaned — SOV-Transcribe-1 at 1.50%. Read parliamentary speech, and the European languages in its training mix are exactly the ones this set draws from.

• Earnings22-Cleaned — SOV-Transcribe-1 at 4.06%. Real earnings calls: names, tickers, numbers, crosstalk. The model's worst surface, and the one closest to enterprise audio.

Google does not publish a component breakdown for Gemini 3.5 Transcribe on the same three sets, so the honest statement is that the composite favours Salesforce by 0.32 points and the per-dataset picture for Google is not publicly visible. Anyone claiming a clean win on, say, call-centre audio is extrapolating from a composite weighted half toward synthetic conversation.

Where the hosted model is simply ahead

Four capabilities on the Google side have no counterpart in the Salesforce release, and each one decides a category of product.

• Languages — 85-plus with automatic detection and mid-session code-switching, against seven European languages. If your audio includes Mandarin, Hindi, Arabic or Japanese, SOV-Transcribe-1 is not a candidate and the accuracy comparison is moot.

• Speaker structure — diarization for up to eight speakers, with attribution beyond three speakers labelled experimental. SOV-Transcribe-1 documents no diarization at all; a two-person call transcript comes back as one undifferentiated block of text.

• Timestamps — word-level output on the batch endpoint. Salesforce's model emits a transcript string, so subtitle generation or compliance search by timestamp has to be built downstream.

• Streaming — a dedicated live endpoint with ten-minute sessions and bidirectional WebSocket transport. SOV-Transcribe-1 is explicitly non-streaming; there is no partial-transcript mode to enable.

There is a fifth difference that is not a feature but a constraint. Google's own documentation flags that word-level timestamps degrade transcription accuracy and are incompatible with custom vocabulary, and that diarization is incompatible with vocabulary biasing too. The hosted model makes you choose which capability you want per request. The Salesforce model makes you choose once, at the level of "no diarization, ever".

Where the open model is ahead, beyond the score

The obvious answer is data residency, and it is the right one for a large share of buyers: Apache 2.0 weights on hardware you control means audio never leaves the building. For call-centre recordings in financial services or healthcare, that is frequently a hard requirement rather than a preference, and no hosted API clears it regardless of WER.

The less obvious answer is biasing. On Contextual Earnings-22, SOV-Transcribe-1 lifts keyword F1 from 71.9 with no hints to 96.8 given the clip's own keywords, and holds 94.0 when handed the full hour's keyword list with most entries as distractors. Gemini offers custom vocabulary biasing too, up to 1,000 terms with best results reported around 100 — but it cannot be combined with the diarization or timestamps you might also want. If entity accuracy is the product, both models can do it; only one of them can do it while also emitting speaker labels.

Style control is the third. SOV-Transcribe-1 was trained to take instructions — verbatim versus cleaned-up, digits versus words, fillers kept or dropped — and Salesforce reports 5.05% preference-aware WER with the instruction given. Google's equivalent is "Smart transcription", which covers filler removal and intent-aware formatting but is a mode rather than an instruction surface.

The cost comparison that actually matters

Google publishes Gemini 3.5 Transcribe at $2.00 per million audio input tokens and $12.00 per million output tokens, which the pricing page converts to roughly $0.003 per minute in and $0.002 per minute out, or about $0.005 per minute blended. Artificial Analysis' board carries the same model at $5.00 per 1,000 minutes. Either way, a thousand minutes of audio costs about five dollars.

SOV-Transcribe-1 has no per-minute price because Salesforce sells no endpoint. The comparable figure is throughput: with native vLLM support and the DFlash draft model, one RTX PRO 6000 Blackwell Server Edition transcribes up to 1,039 seconds of audio per second. At that rate a single card clears a thousand minutes in roughly a minute of wall clock, so the crossover against a $5-per-1,000-minute API arrives at any serious volume — and never arrives at all if you are transcribing three hours a week and would have to rent the card by the hour to do it.

Two caveats belong with that arithmetic. The 1,039 figure is a vLLM development-build measurement from November 2026 with max_num_seqs=64, which is also the setting required to keep startup profiling inside 96 GB once the draft model is loaded. And the integration is a plugin pinned to vLLM 0.32.0, with native upstream support described as work in progress. You are not just buying a GPU; you are maintaining a serving stack.

Decision rules

Reach for Gemini 3.5 Transcribe when any of these is true: the audio is not in one of seven European languages, you need speaker labels or timestamps as first-class output, you need partial transcripts while audio is still arriving, or your volume is low enough that a GPU never pays for itself. The 0.32-point accuracy deficit is a real cost, but it is a cost measured on a composite half-weighted toward synthetic conversation, and it buys four capabilities you would otherwise have to build.

Reach for SOV-Transcribe-1 when the audio is English-dominant and entity-heavy, when the vocabulary is known in advance, when the recordings are under thirty minutes, and when "the audio stays on our hardware" is a line in a contract rather than a preference. Batch overnight transcription of recorded sales calls is the workload this model was clearly designed for.

Mixed estates are the common outcome, and the awkward part is never the transcription. It is that transcription feeds a second model — a summariser, a classifier, a QA agent — and changing that consumer means a new vendor, a new key and a new invoice. OrcaRouter removes that step: 200+ text, image and video models behind one key at provider list pricing with 0% markup, so the model downstream of the transcript can be swapped or failed over without touching the transcription pipeline at all. It does not host either speech model, and it does not need to. The transcription layer stays where it is; the routing begins with the text it produces.

The one thing not to do is treat the 0.32-point gap as the decision. It is the smallest number in the comparison and, for most teams, the least consequential.