Ming-3.0-audio-tiny Runs on One GPU Now — and Its Config Finally Says 128K

The checkpoint behind Ming-3.0-audio-tiny has been downloadable since 3 September 2026 — two safetensors shards, 17.7 GB, pushed nine minutes after the repository itself was created. What did not exist until this week was nearly everything that tells you how to run it. Between 16 and 19 October 2026 inclusionAI replaced the config that shipped in September, turning a video-language-audio variant with an 8,192-token window into an audio-only one that claims 131,072, added the Talker that produces speech, added two reference voice clips with their transcripts, and — after three days in which the repository had no model card at all — wrote one. The code those documents point at arrived on GitHub on 15 October, in a repository named for the bigger sibling, Ming-3.0-audio-flash, and it carries three deployment profiles for the tiny model that the tiny model never had. Nobody announced any of this. So the question worth answering is not what is new, because the weights are seven weeks old and byte-identical to the day they landed. It is what a reader can now run that they could not run last Monday, and what it costs in accuracy when they do. The documented AI failures and agent incidents behind releases like this one are collected in the OrcaRouter AI Incident Archive where each entry carries its own sources, severity, confidence grade and disputed flag.

What changed, dated to the commit

The Hugging Face commit log is short enough to read in full, and the shape of it is unusual: three commits in September, nothing for three weeks, then six in four days.

• 3 September, 11:45 and 11:49 UTC — the repository is created, then the two shards land under "Add files using upload-large-folder tool". The README that comes with them is titled "Ming Audio V3", declares Apache 2.0, tags the repo automatic-speech-recognition, and says the repository "currently documents the ASR path."

• 17 September — the examples/ folder is deleted. After this the repository is weights, custom modelling code and a short card, and nothing else.

• 16 October, 08:14 to 08:20 UTC — the consequential one. config.json is rewritten: max_position_embeddings on the text backbone goes from 8,192 to 131,072, transformers_version from 4.56.2 to 5.14.1, and audio_config.model_type from bailing_moe_v3_vl_audio to bailing_moe_v3_audio — the video-language-audio type that no longer describes the model. In the same push auto_map gains AutoModel and AutoProcessor entries it did not have, a processor_config.json appears, and the talker/ tree arrives: a 1.4 GB speech model, a 1.6 GB VAE, a CAM++ speaker encoder, text-normalisation finite-state transducers for English and Chinese. The old README goes out in the same push — for the next three days the repository has no model card at all.

• 17 October, 11:44 UTC — "Add voice presets." Two reference WAVs, US_James.wav and 灵楷.wav, plus a JSON file holding their exact transcripts.

• 19 October, 10:36, 11:44 and 11:49 UTC — three cleanup and upload commits.

• 19 October, 13:19 UTC — "Add Ming-3.0-audio-tiny model card." 8,514 bytes, and the first document in the repository's history that describes a speech-and-audio model rather than an ASR path.

The GitHub side has its own timeline, and it starts later. inclusionAI/Ming-3.0-audio-flash was created on 15 October at 03:50 UTC — six weeks after the weights existed — and was still being pushed on 19 October at 14:53 UTC. It has eight stars, an MIT licence file, and a deploy/ directory containing six YAML profiles, three of which are for the tiny checkpoint by name. There is no inclusionAI/Ming-3.0-audio-tiny repository; that URL returns 404, and the tiny model's code lives in the flash repository alongside its sibling's.

The config rewrite is the real story

A context-length field going up by 16× is the kind of change that is either cosmetic or structural, and in this case the rest of the diff says structural. In September the repository shipped modeling_bailing_moe_v3_audio.py and processing_bailing_moe_v3_audio.py in the tree, but its config described an audio tower of type bailing_moe_v3_vl_audio — a leftover from a video-language build — and its card told you to instantiate the model with trust_remote_code=False, which is not what custom modelling files want. The October config is the first one where the declared type, the declared runtime, and the code path agree.

• Backbone — 24 layers, hidden size 1,536, 16 attention heads, 128 routed experts with 8 active per token and one shared expert, expert intermediate size 512, first dense layer at index 1, routed scaling factor 2.5, vocabulary 157,184.

• Attention — MLA, with a 512-wide KV latent and a 256-wide query latent, rope_theta 6,000,000, partial_rotary_factor 0.5. The low-rank KV form is why a 131,072-token window is even arguable on one card: the cache stores the latent, not per-head keys and values.

• Audio tower — 32 layers, width 1,280, 20 heads, a 3,000-frame context and 128 mel bins, with a downsampler of kernel [3,3,3] and stride [2,2,2]. Audio token id 157,168.

• Size — 9.3B total parameters, 2.7B activated per token, bfloat16 throughout, declared against Transformers 5.14.1.

• Inputs and outputs — audio plus a text instruction in; text out, or text plus synthesised speech through the optional Talker. Chinese and English, including accents and dialects, per the card.

What the config rewrite does not do is add any weights. The four safetensors files in the repository — two Thinker shards, the Talker, the VAE — have the same sizes at every revision I checked, back to the initial upload. This was a documentation and packaging change to an unchanged checkpoint, which is exactly why the 8,192-to-131,072 jump is trustworthy: nobody retrained anything, so the claim can be tested by loading the model and asking for a long context rather than taken on faith.

One piece of paperwork is still inconsistent. The repository's top-level Hugging Face pipeline tag reads text-to-speech, while the card's own summary calls it an audio-language model that accepts instructions and returns text or speech. The tag is a leftover from the Talker being the most recently touched capability, and it is the sort of thing that matters to nobody except the person filtering the Hub by task.

One GPU for text, two for speech

This is where the tiny model earns its name, and it is only visible because the deployment profiles are now public. The flash model's text profile runs on devices: "0,1,2,3,4,5,6,7" with tensor_parallel_size: 8 — eight 80 GB cards, which is what 256 GB of bf16 weights forces before any cache is allocated. The tiny model has three profiles of its own, and two of them ask for one card.

• Text and ASR, vLLM-Omni — a single stage on devices: "0", tensor parallel size 1, max_model_len 131,072, max_num_batched_tokens 32,768, gpu_memory_utilization 0.92, sampling at temperature 0.6 and up to 65,536 output tokens.

• Full audio with speech output — two stages. The Thinker on devices: "0" at 0.78 of the card, and the Talker on devices: "1" at 0.18, with the Talker stage capped at one output token and temperature 0.0 because it is not generating text.

• ASR reproduction — a thinker-only pipeline with a 10-gram no-repeat logit processor bolted on, audio limited to one clip per prompt, max_model_len 131,072, temperature 0.0, 32,768 output tokens. That is the configuration the vendor's recognition numbers were taken under, and it is worth copying exactly if you intend to compare.

• SGLang — one GPU, one visible device, text output only. The card is explicit that speech output is not available on this path.

Those profiles are files, not prose, and they are visible in the flash repository's deploy/ directory — six YAML profiles, three of them named for the tiny checkpoint.

Read the memory numbers honestly. 0.92 is a share of whatever card you point the stage at, not a requirement: the Thinker's shards total 17.74 GB, so the weights fit inside a 24 GB card with roughly 6 GB left over for cache and activations. The 0.92 reservation is sized for a 131,072-token window plus a 32,768-token batch at once, and it drops if you lower max_model_len. What you cannot do on one card is produce speech, because the Talker and its VAE are a second process with a second weight set — 3 GB before the CAM++ encoder and the transducers — and the reference profile gives them their own device. Text on one GPU, conversation out loud on two: that is the honest arithmetic, and it is a different budget line from the sibling's eight.

What is actually in the download

The repository totals 20.78 GB. The Thinker is 17.74 GB in two shards of 9.55 GB and 8.19 GB; the Talker adds 1.40 GB, its VAE 1.56 GB, the speaker encoder 28 MB, and the tokenizer and normaliser files make up the remainder. There is no quantised variant, no smaller checkpoint, and no GGUF conversion published by the vendor, so the bf16 figure is the floor unless you quantise it yourself.

• Loading — AutoProcessor.from_pretrained(..., trust_remote_code=True, use_fast=False) and AutoModelForCausalLM.from_pretrained(..., dtype=torch.bfloat16, device_map="auto", trust_remote_code=True). The card's own snippet then casts audio_feats to the model dtype by hand, which is a small tell that the processor path was not the priority.

• The ASR contract — the validated prompt asks the model to name the spoken language and put a tab between the language label and the transcription. If you are building an evaluation harness, that tab is your parser. A generic "transcribe this" instruction is not the configuration inclusionAI measured.

• Voices — the vLLM-Omni default is a built-in voice called 灵小柔, and the model directory ships two reference presets, 灵楷 and US_James, with transcripts. Voice cloning takes a reference WAV plus its exact transcript; the card warns that you cannot combine a built-in voice selection with a reference clip in one request.

• Licence — MIT in the card's metadata and in the GitHub repository's LICENSE, which is dated 2025. Note what is missing: the Hugging Face repository contains no LICENSE file of its own. The only licence text physically inside it belongs to a bundled language-identification model under talker/data/. The MIT claim is real but it lives one click away, in a repository named after the other model.

• Traction — three downloads and five likes, as of this week. The flash model has 22 downloads and eight likes. Neither number is evidence about quality; both are evidence that a September upload with no card and no announcement did not travel.

The benchmark picture, and the column that is missing

Three PNGs in assets/ carry the evaluation, and all three are inclusionAI's own run. The vendor marks self-evaluated entries with an asterisk and notes that comparison models were reproduced under its own prompts and decoding settings, so their numbers may differ from their official reports. No independent evaluation of this checkpoint has been published that I could find — no arena entry, no third-party score — which means every figure below is a vendor describing its own model. That is not a reason to ignore them; it is a reason to read them as a design brief rather than a verdict.

The useful comparison is tiny against its own sibling, because both ran the same harness. On recognition the smaller model gives up a little and on long-audio reasoning it gives up a great deal.

• Chinese ASR average — character error rate 3.79 for Ming-3.0-audio-tiny against 3.45 for Ming-3.0-audio-flash. Lower is better, so tiny is 0.34 points behind.

• English ASR average — 4.36 against 3.69. Dialect average — 10.29 against 9.40.

• BigBenchAudio — 99.20 against 99.40. Effectively a tie, and the one row where the size difference does not show.

• MMAU — 75.90 against 82.00. MMAR — 67.70 against 74.40. MMSU — 73.06 against 78.82. MMAU-Pro — 56.42 against 66.41.

• Long-audio accuracy — 56.39 against 66.30. LongAudioSpan — 35.45 against 41.12. Long-audio Chain score — 10.56 against 12.31.

• Spoken interaction — URO-Bench-Pro 54.15 against 81.92, a 27.8-point collapse. VoiceBench 74.29 against 86.74. SpeechFCEval 69.80 against 82.60.

• Tool calling — BFCL-v3 Audio 79.80 against 81.40. Music — MuchoMusic 58.07 against 65.65.

The dialect rows are the interesting exception, because tiny does not lose uniformly. It beats its bigger sibling on Anhui (5.42 against 5.76), Northeastern Mandarin (4.21 against 4.28), Jiangsu (11.44 against 12.15), Shaanxi (5.81 against 6.00), Shandong (10.17 against 10.69) and Yunnan (11.28 against 11.53), and loses badly on Hakka (24.08 against 16.51), Hokkien (14.15 against 12.84) and Shanghainese (12.49 against 11.15). A 2.7B-active model holding six of fifteen dialect rows against a 6.5B-active one is the most favourable thing in the entire evaluation, and it is buried in the middle of a table where the average hides it.

Two gaps are worth stating because they are checkable and the card does not point them out. First, five rows in the recognition image carry no tiny value at all — the speaker-diarization and concatenated minimum-permutation rows for AISHELL-4, AliMeeting and AMI are printed as a dash. Whatever the tiny model does on "who spoke when" is unpublished. Second, the text-intelligence image has four columns and none of them is the tiny model: its columns are Ming-3.0-audio-flash, Qwen3.8-Omni Flash, Qwen3.5-Omni Plus and Qwen3.5-Omni Flash. The card's evaluation section says the figures below include results for both Ming variants, which is true of the two audio images and not true of the text one. If you need to know whether the tiny backbone can hold a reasoning conversation, inclusionAI has not told you.

What the week actually bought you

Before 16 October, a reader who wanted to test this checkpoint had a 17.7 GB download, a config describing a video-language model with an 8K window, and a card instructing them to load custom code without enabling custom code. After 19 October they have a config that matches its own modelling files, a 128K claim that can be tested against unchanged weights, three named deployment profiles including a single-GPU one, a voice pack with transcripts, and a card that says what the model is for. None of that is a new model. All of it is the difference between a checkpoint on a disk and a checkpoint someone can evaluate in an afternoon.

The cost is the part the sizing hides. On the vendor's own tables, moving from the 6.5B-active model to the 2.7B-active one buys you an eighth of the GPU count and charges you 27.8 points of URO-Bench-Pro, ten of VoiceBench, and roughly a fifth of the long-audio Chain score. That is a reasonable trade for a transcription service, a bad one for anything that has to reason across an hour of audio, and the vendor printed both columns next to each other so you can decide which one you are building.

Where a router earns its place, and where it does not

OrcaRouter does not serve Ming-3.0-audio-tiny. It is not in our catalogue, we cannot give you an endpoint for it, and nothing here should be read as an offer to. What a router is good for in a pipeline built around a checkpoint like this is the half that is not the checkpoint. Speech systems are mostly text work: the prompt that decides what to ask the audio, the parser that turns a tab-delimited language label into structured output, the summary written from the transcript, the classification that decides whether an audio file goes to the 2.7B model or the 126B one. Those are ordinary text calls, and one API for more than 200 models at 0% markup — provider list price passed through, so a vendor price cut is live on our side the same day — is a simpler thing to wire than a stack of vendor contracts. Automatic failover is what lets you put a newly documented model behind part of a pipeline without betting the whole path on it.

The boundary is worth stating plainly. A 17.7 GB checkpoint you run on your own GPU is infrastructure, not an API call. A router routes calls. It does not turn an open-weights audio model into a hosted one, and it does not turn three downloads into production evidence.

What to watch

Three things would change this picture, and all three are checkable. First, an independent score: the moment an arena or a third-party evaluator lists this checkpoint, the 27.8-point spoken-interaction gap stops being a vendor's self-report and becomes a data point; until then, note that the same vendor also published six dialect rows where the smaller model wins. Second, a LICENSE file inside the Hugging Face repository, which would move MIT from a link in a card to a fact in the artifact a procurement process actually reads. Third, a tiny-specific GitHub repository: today the deployment profiles that make the single-GPU claim real live in a repository named after the bigger model, which is fine until someone renames it.

Until one of those lands, the accurate description is unglamorous. Same weights as 3 September, roughly 18 GB of Thinker, a 131,072-token window that arrived in a config rather than in a checkpoint, three serving profiles that fit on one card for text and two for speech, MIT on paper, five reference rows it declined to publish, and a model card that is three days younger than the config it describes. The week did not make the model better. It made it legible, and for an open-weights checkpoint nobody announced, legible is the whole product.