AuK: Tencent Quietly Open-Sourced a 1.5B Speech Model That Treats Every Audio Task as a Text Command

Here is a sentence no single speech tool you can download today will do in one pass: take this recording, swap the word "house" for "home", raise the pitch by a semitone, take out the breath before the last phrase, scrub the background noise, and then re-read the result in a brand-new voice described only as "a warm middle-aged man with a slight Cantonese accent". Tencent's answer to that sentence is AuK, a 1.5-billion-parameter "foundation model for speech generation and editing" that went open-source this week with no launch post and no press release — and whose distilled sibling AuK-Flash now comes with the one thing this story was missing when we first wrote it up: a technical report with numbers attached. "AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing" (arXiv:2609.08936, cs.SD, submitted 2026-09-08) is now cited on both the Hugging Face model card and the GitHub README, with the paper listing dated 2026-09-08 and the code repository's own news line dated 2026/09/09. What the report does not do is settle the question that mattered — every figure in it is Tencent's own, run against baselines Tencent picked, and as of 2026-09-10 nobody outside the company has reproduced a single one.

This is a what-we-know-so-far piece, revised once. Tencent still has not announced AuK on any of its main channels that we can find, so the record is: the Hugging Face model cards and repositories for AuK and AuK-Flash, the code repository under the Tencent-Hunyuan org, the project site at auk-project.github.io, and now the paper. That last addition changes what is knowable — the architecture, the training recipe and the evaluation numbers are described in detail for the first time — without changing what is verified. Treat everything numeric below as vendor-reported, because that is exactly what it is, and note that the report's comparisons are entirely against other open models, not the closed hosted speech platforms AuK would actually compete with.

What actually shipped, and how quietly

The timeline is still the most honest summary of the release, with one correction from our first pass. The Hugging Face repositories were created in mid-August — AuK on 2026-08-18 — but sat dark until this week. When we read the AuK model card on 2026-09-09 its single news line was dated 2026-09-07; a day later that same line reads 2026/09/09 and points readers at the paper and the demo Spaces. The GitHub repository, which holds the inference and training code, was created on 2026-08-19 and last pushed on 2026-09-09. In other words: weights, then code, then a technical report — three staged drops in four days, and still no announcement from Tencent's main channels as of writing.

• The weights live on Hugging Face under the tencent org and are mirrored on ModelScope under Tencent-Hunyuan — two mirrors, same MIT licence.

• Each model repo holds a single safetensors checkpoint plus a config and a VAE: auk_base.safetensors at 6.12 GB and vae.safetensors at 0.64 GB for AuK; the same layout for AuK-Flash. The model card asks you to also download Qwen/Qwen2.5-Omni-3B, the text encoder AuK loads separately at runtime.

• The "nobody noticed" framing has expired, and quickly. The code repository sat at zero stars and zero forks when we checked on 2026-09-09; by 2026-09-10 it shows 216 stars, 12 forks and its first open issue. The AuK card reports about thirty downloads in the last month with 26 likes. What has not changed: the discussion tabs are empty, and the card still notes the checkpoint is not deployed by any inference provider.

• Both model cards, the README and the paper link demo Spaces on Hugging Face and ModelScope. The Hugging Face Space was not publicly reachable without sign-in when we checked, so treat the "try it in your browser" path as unconfirmed for anonymous users.

The Hugging Face card above is still the whole public surface of an installable release: a model card, a config, two safetensors files and a licence. What has been added since is the documentation layer around it — a paper, a benchmark table and a citation block — and none of that changes the fact that there is no changelog, no discussion thread and no inference-provider listing behind it.

The pitch: one instruction interface, sixteen speech tasks

AuK's claim is not that it is the best text-to-speech model ever released. It is that generation is only one of five families of speech tasks the model was trained to do through a single natural-language interface, where a request is a chat message that can carry text plus an optional reference audio file. The paper formalises exactly the taxonomy the project site was already advertising:

• Speech generation — zero-shot TTS (speak this text in the voice of the reference clip) and instruction TTS (generate speech from a voice description alone, with no reference audio at all).

• Content editing — rewrite what was said: replace, insert or delete words in an existing recording; and lyric editing, which rewrites sung lyrics while preserving the melody and the voice.

• Acoustic editing — pitch in semitones, speaking-rate scaling, and volume in decibels.

• Paralinguistic editing — emotion changes, timbre changes from a description, de-accenting, adding or removing nonverbal sounds (breaths, laughs, coughs), and converting between normal speech and whisper while keeping the speaker.

• Enhancement and separation — speech denoising and dereverberation, speech separation by talking order, music/vocal separation, and target-speaker extraction identified by what the speaker says.

The instruction-TTS case is the one that separates this from a typical voice-cloning release: AuK will read a sentence in a voice that exists only as a text description — the docs' own example specifies a woman in her twenties speaking to a partner who has just come home, warm, caring and lightly coquettish, with a soft sweet timbre and a slightly rising sentence-final tone, all with no reference clip attached. That removes the need for a reference recording, which is the usual gating cost of cloning. The report puts a number on that task for the first time, and it is one of the few places AuK beats the field outright rather than edging it.

Inside the "1.5B": the number undersells the runtime

AuK's parameter count describes the diffusion transformer, not the whole pipeline, and the paper now spells out both halves. The backbone is a hybrid rectified-flow Transformer — thirty layers made of ten dual-stream MMDiT blocks followed by twenty unified single-stream DiT blocks, hidden size 1536, 24 attention heads of dimension 64, SwiGLU intermediate width 3072, which is where the "roughly 1.5 billion parameters" figure comes from. Around it sit two separately loaded components: a multimodal LLM (Qwen/Qwen2.5-Omni-3B) that turns the instruction and any reference audio into semantic conditioning, and a causal 24 kHz audio VAE — the config names it BigVGANFlowVAE — that runs 64-dimensional latents at a 50 Hz frame rate, with encoder channel widths up to 768 and a decoder up to 1536. So the full runtime is the ~1.5B backbone plus a 3B encoder plus the VAE, and the download instruction is explicit that the encoder is a separate checkpoint with its own terms.

Training, per the report, ran in stages and at a scale the project site only hinted at: a generation-only warm-up of 50,000 updates, then 600,000 joint generation-and-editing updates, on roughly 3.03 billion instruction–audio instances representing about 1.95 million hours of effective supervision, across 256 GPUs, with a further 1.24 million updates on the VAE. Post-training combined human-feedback preference optimization with reward-based reinforcement learning, built on 818 informative preference groups and 9,080 rated candidates, RL prompt pools of 10,000 zero-shot and 5,000 instruction-following prompts, 30,000 style-judge samples, and a reward model fine-tuned from Qwen2.5-Omni-7B. That is a serious post-training stack for a 1.5B open release, and it is also a reminder that "open weights" describes the artifact, not the compute that produced it — a point worth holding onto when weighing whether you could reproduce it.

Every figure in that spec sheet is read from Tencent's own repositories and website rather than measured by us, and the paper did not change that — it just moved more of those lines from "claimed" to "documented". The one number that has actually been tested since we first published is the repository's own activity, which went from zero stars to a few hundred in a day.

The project site remains where the architecture diagram and the academic co-branding live, and it is still the cheapest way to see the shape of the model: a multimodal language model for semantic conditioning, a 50 Hz VAE for acoustics, and a hybrid transformer under a flow-matching objective. Its benchmark section, which listed an evaluation apparatus with no scores when we first looked, now has a paper to defer to.

The technical report lands, and the numbers are Tencent's own

This is the substance of the update. The paper reports results across seven evaluation suites — the same list the project site had been advertising without numbers — and on most of the generation and content-editing axes AuK leads the open field. Read these as a vendor benchmark table, because that is what they are: tencent ran every comparison, chose every baseline, and has published no code for reproducing the harness yet.

• Zero-shot TTS on Seed-TTS-Eval: AuK averages 2.65 WER with 0.795 speaker similarity, against Qwen3-TTS at 3.07 and 0.745, Seed-TTS at 3.65 and 0.778, and VoxCPM2 at 3.65 and 0.767. AuK-Flash lands at 2.85 and 0.790. The best single splits are 1.02 WER on the English test set and 5.91 on the Chinese hard set.

• Instruction-controlled TTS on InstructTTSEval: AuK scores 83.37 on Chinese and 81.60 on English description-following, versus Qwen3-TTS-VD at 81.10 and 82.40 and MOSS-VoiceGenerator at 80.00 and 82.00. AuK-Flash is 78.80 on Chinese but ties the field's best English score at 82.40.

• Content editing on SpeechEditBench: 91.83 accuracy against Ming-UniAudio's 76.46, and 71.33 on prosody against 26.50 — the two largest gaps anywhere in the report.

• Instruction-guided editing on Ming-Freeform-Audio-Edit, full setting: word error drops from 10.46 to 3.09 on Chinese and from 14.28 to 3.96 on English versus Ming-UniAudio, with edit accuracy rising from 79.62 to 91.47 and from 70.98 to 85.25. On acoustic edits the same comparison moves word error from 5.01 to 2.02 in Chinese and 10.75 to 3.48 in English.

• Paralinguistic editing on MMAE-Speech: instruction-following rate 48.23 and content-rewrite 88.11 against Step-Audio-EditX at 43.52 and 77.27, and Ming-UniAudio at 34.13 and 76.01. AuK-Flash posts the best edit-model recall in the table at 13.85.

• Restoration, where the model is competitive rather than dominant: DNS Challenge desynchronised word error 2.66 with 0.99 speaker similarity against RE-USE at 3.31; CHiME-4 word error 7.98 against RE-USE at 10.71; Libri2Mix word error 9.12 against MossFormer2-SS at 9.34; and VCTKSR word error 3.06 with 0.97 similarity.

• The VAE itself, evaluated separately: AuK-VAE reaches 4.143 PESQ on speech, 3.833 on general audio and 3.884 on music, against MiniMax-H3-AudioVAE at 3.633, 2.835 and 2.801 — a clean sweep that matters more than it looks, because a lossy acoustic latent caps every task built on top of it.

The pattern across those bullets is more interesting than the headline wins. AuK is far ahead on anything that means "change what is said or how it sounds, and keep everything else" — content editing, prosody, acoustic edits, reconstruction. It is much closer to the field on emotion and paralinguistic editing, where its SpeechEditBench emotion accuracy of 9.94 is barely distinguishable from Ming-UniAudio's 3.43 on a benchmark where both are weak, and its overall acoustic score of 37.07 sits in the same range as the baselines. So "one model for every speech task" is Tencent's framing; what the report actually shows is one model that is excellent at several of them and merely present for the rest.

Two checkpoints: AuK and AuK-Flash

Tencent shipped two variants under the same MIT licence. AuK is the full-quality base model, and the report pins its sampling settings at 32 function evaluations with a classifier-free-guidance scale of 2.0. AuK-Flash is the distilled version: consistency initialisation and a task-routed Decoupled DMD objective, generating in four fixed steps with guidance switched off entirely, which the paper reports as a 4.5× wall-clock speedup over the full model under matched conditions. The paper's own numbers keep Flash close on most generation tasks — 2.85 average word error and 0.790 similarity on Seed-TTS-Eval — which is what makes the speed claim worth testing rather than dismissing: four-step diffusion at near-teacher quality is what would make a 1.5B speech editor feel interactive on a single GPU. That said, "matched conditions" is Tencent's phrase describing Tencent's benchmark, and a 4.5× number measured by the authors is exactly the kind of claim that needs a third party before it changes any procurement decision.

What you can run today

The practical surface is broader than a typical quiet weight drop, because the code landed with it. Installation targets Python 3.10 via uv or Conda, and the README offers extra install targets that pull in Gradio, ComfyUI nodes, or the fine-tuning stack. The command-line tool auk-infer covers every task with the same message shape: an --instruction is always required, and --audio is optional depending on the task. A Gradio server (auk-gradio) exposes the models with a selector, and there are ComfyUI custom nodes with a reusable workflow, though the integration is documented with a 30-second source-plus-target sequence limit. A Python API mirrors the CLI, and a lightweight fine-tuning pipeline trains on JSONL of instruction-plus-audio pairs using the same message format as inference. The README also describes a Prompt Enhancer that turns free-form requests into ready-to-run auk-infer commands; it expects an OpenAI-compatible chat endpoint and falls back to a local SenseVoiceSmall model for speech recognition when no cloud ASR credentials are set. One current limitation is documented plainly: Qwen3-Omni is not supported as the encoder path yet, so the Qwen2.5-Omni-3B checkpoint remains the one to download.

What the docs still do not give you is a hardware floor. No VRAM figure is published for inference. The config files carry a gradient-checkpointing note about a ~91 GB peak dropping to ~75 GB, which describes the training-time memory envelope rather than a realistic single-shot inference number, and the reference command that loads AuK and AuK-Flash together spreads them across two GPUs — while noting both can share one. Budget for the download itself: roughly 6.8 GB per variant plus the Qwen2.5-Omni-3B encoder, then measure memory on your own GPU.

What is not confirmed

Being precise about the unknown is still the point of covering a model this early, and the report removed exactly one item from this list while leaving the rest intact:

• No independent run exists that we can find. No third-party voice samples, no benchmark replication, no discussion-thread impressions. The repository's first open issue is filed and unanswered, and the model card's download count is still in the dozens, so the gap between a published table and a reproduced one is currently the entire story.

• The evaluation is self-run and narrow in one specific way. Every baseline in the report is another open-weight model — Qwen3-TTS, Seed-TTS, VoxCPM2, Ming-UniAudio, Step-Audio-EditX, MOSS-VoiceGenerator, RE-USE, MossFormer2-SS. None of the closed hosted speech platforms a buyer would actually weigh AuK against appear, so "leads the field" here means "leads the open field Tencent selected".

• The name "AuK" is still never expanded anywhere in the paper, the cards or the code, and it collides hard in search with the seabird, American University of Kuwait and unrelated repositories.

• The team is now at least visible — the paper carries 33 authors led by Ziyang Ma, with affiliations spanning the Tencent Hunyuan org, Shanghai Jiao Tong University, the Shanghai Innovation Institution and Nanyang Technological University — but who inside Tencent owns the model day-to-day, and whether it is a Hunyuan line or a separate academic collaboration, remains unstated.

• Language coverage is still only partly documented: the AuK-Flash card declares Chinese and English and the code examples mix both, but the base model has no formal language list. Licensing has the same shape — AuK itself is MIT, while its separately-downloaded Qwen encoder carries its own terms, so "MIT model" and "MIT everything you need to run it" are not the same statement.

Why a unified open-weights speech model matters

Read AuK's task list against what a builder actually does today and the bet becomes clearer than it was before the numbers arrived. Doing all of those jobs with existing tools means chaining several specialist models — one open checkpoint for cloning, a different one for enhancement, a separate separator, and manual or model-assisted editing for content — or renting several closed hosted APIs, each priced per task and per minute of audio, and none of them editable in the way AuK describes. A single open-weight checkpoint that treats all of those as one text-conditioned generation problem is a genuinely different object, and the report now supports a sharper version of that claim than the marketing did: the editing half is real, not aspirational. Voice cloning got commoditized over the past year, but instruction-driven content and prosody editing is where the closed hosted platforms still earn their margin, and a 91.83 against 76.46 on content editing is the first evidence that an open model can take that ground.

The honest qualification is that this is a diffusion model with a 3B language model bolted on for conditioning, and it inherits that shape's costs. Quality per task is uneven — excellent where the task is "preserve everything except the edit", thinner on emotion — and there is no hosted endpoint, no batching story and no published latency for anything but the Flash variant's internal comparison. For the pieces of a voice pipeline around it — the LLM that decides what should be said, and the Prompt Enhancer's OpenAI-compatible chat endpoint — a routing API is the natural fit, and OrcaRouter fronts 200+ models at provider list price with no markup, so that half of the stack needs one key and no second contract. The audio half is still a GPU you control and a checkpoint nobody outside Tencent has audited.

Who should look now, and who should wait

For speech researchers, tool builders and anyone whose job is evaluating new voice models, the case for downloading AuK this week is stronger than it was on day one and still comes with the same caveat. You now get a documented architecture, a reproducible-looking recipe and a full table of targets to beat — which is exactly what makes an unreproduced claim worth attacking. The cost of being early remains a weekend of setup and a GPU. For anyone choosing a production voice stack, the report changes the shape of the decision without settling it: there are now numbers to argue with, but they are the vendor's, they exclude the closed platforms you would actually benchmark against, and there is still no API, no VRAM floor and no track record. Watch, in order: an independent replication of the Seed-TTS-Eval and SpeechEditBench rows; published samples or a reachable demo Space; and an inference provider or hosted API picking AuK up, at which point the pass-through price on our side is the provider's list price, same day, because that is what 0% markup means. The interesting question is no longer whether Tencent shipped another TTS model — it is whether one open model can really hold all five of those task families at once, and now that Tencent has published its own answer, the only thing left is for someone else to check it.