Ming-3.0-audio-tiny vs Sesame Preview: One Can Speak and Won't Be Scored, the Other Speaks for a Living

The odd thing about putting Ming-3.0-audio-tiny next to Sesame Preview is that the obvious axis — voice quality — is the one neither side will let you measure. Ming-3.0-audio-tiny ships a speech generator: a 1.4 GB Talker, a 1.6 GB VAE, a CAM++ speaker encoder, three named voices and a reference-clip cloning path. And its vendor publishes no text-to-speech benchmark for any of it. The three evaluation images in the repository cover recognition, audio intelligence and text. There is no speaker-similarity row, no word-error row for synthesised speech, nothing. Sesame Preview is the mirror image: a product whose entire reputation is the sound of its voice, with no model identifier, no API and no score attached to the voice in the app. The documented AI failures and agent incidents behind releases like this one are collected in the OrcaRouter AI Incident Archive where each entry carries its own sources, severity, confidence grade and disputed flag.

Both are quiet in the same direction and for different reasons, and that is the honest starting point. Ming-3.0-audio-tiny is a checkpoint you can download — the weights have been on Hugging Face since 3 September 2026, and what changed between 16 and 19 October was the paperwork around them: a corrected config, the Talker tree, voice presets, deployment profiles, and on 19 October the first model card. Sesame Preview is an app you talk to, updated in public, described by the company in its own words, and never reduced to a file. One is unmeasured because its vendor chose not to measure it; the other is unmeasurable because there is nothing to run.

What the tiny model's speech path actually is

It is an attachment, not the model. The Thinker — 17.74 GB across two shards, 9.3B parameters total and 2.7B active — reads audio and writes text. The Talker is a second process that turns the Thinker's hidden state into speech, and the reference deployment gives it its own GPU. The text profile runs on devices: "0" with tensor parallelism of one; the full-audio profile puts the Thinker on GPU 0 at 0.78 of the card and the Talker on GPU 1 at 0.18. The SGLang profile is text-only and the card says so plainly.

• Voices — the vLLM-Omni default is a built-in IP voice called 灵小柔; the repository also ships 灵楷 and US_James as preset reference voices under voice_presets/voice_audio/, with matching transcripts in voice_prompts.json.

• Cloning — a reference WAV plus its exact transcript. The card warns that you cannot combine a built-in voice selection with a reference clip in a single request.

• Languages — Chinese and English, "including varied accents and dialects", with no separate language list for the Talker. Whether it speaks English with an American accent as well as it reads it is not stated anywhere.

• Licence — MIT on the weights; the voice presets are the part to read carefully, because a reference clip carries whatever rights its speaker had. The repository ships no separate terms for the two bundled voices.

• Evidence — none for the speech path. The recognition and reasoning numbers are inclusionAI's own; the synthesis side has no published number at all.

That last line is the reason this comparison is short on verdicts. A vendor that measures its ASR to two decimal places and publishes nothing about its T​TS has told you which half of the model it is confident in.

Sesame's side of the same ledger

Sesame's public position is the inverse: a voice good enough that people assumed a human was on the line, delivered through an assistant with memory, connectors, browsing and, since an October 2026 update, the ability to coordinate other agents. The company described that update as a new Conversational Speech Model with better expressivity and pronunciation. It did not publish a checkpoint for it, a model ID, latency figures, or a licence. There is no endpoint, so there is no request to make and no response to score.

The one artefact a developer can hold is sesame/csm-1b on Hugging Face: Apache-2.0, English-only, roughly 1.1 billion parameters, a Llama-family backbone predicting a first audio codebook with a smaller depth decoder for the rest, decoded through Kyutai's Mimi codec at 24 kHz. It went up in March 2025 and has been downloaded about 130,000 times in the last month with roughly 2,500 likes. The card is candid that it is a base generation model — it can produce varied voices, it is not tuned to any particular one, and the interactive demo is a derivative rather than the checkpoint itself.

So the fair statement is: you can hold Sesame's architecture from 2025. You cannot hold the voice that made the company famous in 2026.

Two different jobs wearing the same word

Strip the word "audio" out of both descriptions and the mismatch becomes obvious. Ming-3.0-audio-tiny is a listening model that can also talk. Its evaluation is dominated by comprehension: MMAU 75.90, MMAR 67.70, MMSU 73.06, BigBenchAudio 99.20, Chinese ASR average 3.79 character error, dialect average 10.29. Those are the numbers its vendor chose to publish, and they are all about understanding. Sesame Preview is a talking product that also listens, and its reputation rests on turn-taking — interruption, backchannel, the timing that makes a conversation feel like one.

Those are not the same skill and they do not trade off against each other. A model can be excellent at "what did the third speaker say about the invoice" and mediocre at sounding like a person, and the reverse is equally possible. Nothing in either vendor's material lets you rank them on a shared axis, because they never measured the same axis.

Where the asymmetry bites hardest is the cheap experiment. With Ming-3.0-audio-tiny you can synthesise the same sentence fifty times, vary the reference clip, and listen — an afternoon of work, no permission needed. With Sesame Preview the equivalent experiment is opening the app and forming an opinion, because there is no second sample to draw.

The one Sesame publication worth borrowing

In August 2026 Sesame released TurnBench, an open benchmark for turn-taking in spoken dialogue: about 30 hours of triple-annotated dyadic speech, a scoring protocol borrowed from conversation analysis, a public leaderboard and results for 14 systems, built with Carnegie Mellon, Mundo AI and Oto. This is the most useful thing in the entire matchup, and it is useful precisely because it measures the quality that word-error and preference-Elo boards both miss.

Use it as an instrument rather than as a scoreboard. Sesame's own app is not on it, and Ming-3.0-audio-tiny certainly is not. But if you are considering the tiny model for anything conversational — and its Talker plus its 131,072-token window make that a reasonable thing to consider — then interruption handling and backchannel timing are what you should be scoring, and TurnBench gives you a protocol for doing it instead of a vibe.

The corollary is uncomfortable for both vendors. A model can ship a speech generator and a 128K context and still be terrible at the thing Sesame is known for, and neither dataset in this comparison would tell you.

Cost, once you stop comparing rate cards

Sesame Preview has no price because it has no product to sell at a price: the app is free, the company says it runs no ads and does not sell data, and there is no published tier. Ming-3.0-audio-tiny has no rate card either, but it has something better — an arithmetic floor. Text and ASR run on one GPU. Speech output runs on two, because the Talker and its VAE are a second weight set with its own device. If you only ever need the listening half, the second card is money you do not spend, and unlike a subscription it does not scale with how many sentences you generate.

That is the one place these two are genuinely comparable, and it is a comparison of shapes rather than of numbers: a fixed hardware bill you can predict, against a free product whose terms you cannot audit and whose capacity you cannot buy.

OrcaRouter sits outside both. We do not serve Ming-3.0-audio-tiny — it is a checkpoint you run, not an endpoint we host — and there is no Sesame model to route in the first place. What a router does reach is the text half of a voice pipeline: the script, the translation, the summary, the classifier that decides which recording needs a human. One key for more than 200 models at provider list price with no markup added is a shorter contract than three, and automatic failover keeps a long synthesis batch from dying because one provider had a bad afternoon. None of that makes either model better at being a voice.

What would change the answer

Three events, each checkable. First, inclusionAI publishing a text-to-speech evaluation for Ming-3.0-audio-tiny — naturalness, speaker similarity to the reference clip, error rate on synthesised speech — would turn the Talker from an unmeasured feature into a candidate. Second, Sesame releasing the checkpoint or an API for the production voice would make the app's reputation testable, and the comparison would finally be between two runnable things. Third, an independent party scoring either model on TurnBench would settle the one quality both vendors gesture at and neither publishes.

Until one of those lands, the accurate description is unglamorous. Ming-3.0-audio-tiny is a 17.74 GB listening model with an optional 3 GB speech attachment, MIT weights, a corrected config, a card that is four days younger than the config it describes, and no published number for the half that talks. Sesame Preview is a voice you can hear, admire and not obtain. The choice between them is not a choice between two models. It is a choice between measuring something yourself and not being allowed to.

If you have to decide this week

For transcription, long-recording comprehension, or anything where the output is text you will parse, Ming-3.0-audio-tiny is the only one of the two that is even a candidate — budget one GPU, use the validated ASR prompt with its tab delimiter, and set the context to the length you actually need. For a consumer voice experience you are prototyping, Sesame is the benchmark to hold your work against, not a dependency to build on, because there is no dependency to install.

And for the speech output of the tiny model: try it before you plan around it. Two GPUs, a reference clip, a transcript that matches that clip exactly, and an hour of listening will tell you more than the entire benchmark folder — because the benchmark folder does not contain a single number about it.