Tencent UI-Mate-27B: The Quiet Open-Weight GUI Agent Claiming a WindowsAgentArena Top Spot

On August 14, 2026, Tencent's HY Frontier Multimodal Agent Team pushed three checkpoints to Hugging Face under the tencent/ organization — UI-Mate-27B, a smaller UI-Mate-9B, and the demonstration-guided UI-Mate-democua-27B. Four days later there is still no announcement anywhere: no launch post, no press release, no product tweet. What shipped instead is unusually complete for a silent drop: a project page, a GitHub repository with a working harness, and an arXiv paper submitted two days ago that calls the larger model a new open-weight state of the art in desktop GUI control.

Everything in this piece comes from the model cards, the GitHub repo, the project page, and that paper — none of it has been independently reproduced yet. The checkpoints have zero downloads on Hugging Face as of this writing, which means we are likely among the first people outside Tencent to take them seriously on paper. Here is what is actually knowable from the repos, and what stays unconfirmed until someone else runs it.

Contents

• What shipped, and what hasn't been announced

• What UI-Mate-27B actually is

• The feature that's different: demonstration-guided execution

• The numbers — all of them vendor-reported

• Where those numbers would sit — if they hold

• How to run it

• The stack around a new GUI agent

• What to watch next

• FAQ

What shipped, and what hasn't been announced

Tencent published UI-Mate-27B (27 billion parameters, Apache-2.0, safetensors in BF16) on August 14, 2026, alongside the 9B sibling and the demonstration-guided checkpoint UI-Mate-democua-27B. The two 27B checkpoints are built on Qwen3.6-27B — itself an Apache-2.0 multimodal model released in April 2026 — which means the entire stack, base and fine-tune, is commercially usable. The repos carry last-modified timestamps from this week — the demo-guided checkpoint was touched as recently as today — so Tencent has been actively working on them.

What is public so far:

• The UI-Mate-27B, UI-Mate-9B, and UI-Mate-democua-27B weights on Hugging Face, each with a model card, evaluation tables, and a serving recipe.

• A project page at ui-mate.github.io with a demo video, a usage guide, and a macOS Apple Silicon app (DMG v0.2.4).

• The GitHub repository (github.com/Tencent/UI-Mate) containing the official prompt, response parser, and interaction harness — the card is explicit that this is "an agent checkpoint rather than a standalone visual-chat model" and the harness is recommended for anything real.

• An arXiv preprint (2608.15930), submitted August 16, titled "UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrations."

What is not public yet: Tencent itself has said nothing — this is a repo-first release, not a launch. The demonstration-guided variant that the project page had listed as not yet public now has live weights on Hugging Face: the tencent/UI-Mate-democua-27B repo, created on the same August 14 push, is explicitly tagged as the family's demonstration-guided checkpoint — a distinct model, not a rebrand of the 27B. There is still no hosted API anywhere. That matters: the only way to run UI-Mate today is to download the weights and serve them yourself.

What UI-Mate-27B actually is

UI-Mate-27B is a foundation GUI agent for "long-horizon work across applications and operating systems." It observes live screenshots, reasons over the visible state, and emits structured keyboard and mouse actions for native desktop interaction. Training is supervised fine-tuning followed by online reinforcement learning in executable GUI environments — the model learned by actually driving desktops, not from static screenshots alone.

The action space is the part developers will care about: mouse, keyboard, scrolling, waiting, user interaction, and task completion, with outputs compatible with pyautogui. The model reasons in a normalized 1000×1000 coordinate space and the reference agent rescales its predictions back to the real screenshot resolution, so actions survive display-scaling differences between machines. The model's own README recommends serving it with vLLM behind an OpenAI-compatible endpoint.

The feature that's different: demonstration-guided execution

Most GUI agents take one input: an instruction. UI-Mate takes a second one that is genuinely rare in an open checkpoint — a demonstration. "Show the workflow once. Let the agent adapt it to the task at hand," as the project page puts it.

The mechanism is not video-in-context or a fixed action script. A demonstration records screenshots immediately before and after each keyboard or pointer action, and the pipeline then normalizes the trace into a consistent action-and-frame representation, annotates it with observations, intent, actions, and verification evidence, and segments it into named subtasks with completion criteria. At inference time that becomes a compact workflow injected for the active subtask — but the live screenshot stays authoritative throughout, so when the application state differs from the demo, the model re-plans rather than replaying.

Tencent has made the checkpoint behind this workflow its own public model: the UI-Mate-democua-27B card pairs instruction-only and one-demonstration results on the same evals, and its tool schema adds a subtask_complete action — the mechanism behind those named subtasks. On the OSWorkerBench self-demo subset, one demonstration lifts strict success from 17.17 to 35.35 and progress from 67.85 to 81.14; on the OSWorld subset, progress goes from 40.27 to 65.75; on the GameDev eval set, the average score goes 76.76 to 81.15 while the average trajectory length drops from 303.6 to 253.1 steps — the demo shortens the task as well as improving it. The card reports the demonstration improved 28 of the 33 self-demo tasks and solved four tasks that score zero without guidance. The caveat is the same one that runs through this whole article: these are the team's own paired evals, averaged over three to five runs, and nobody has reproduced them.

The numbers — all of them vendor-reported

Instruction-only execution, straight from the model card:

• OSWorld-Verified average score — 77.0

• WindowsAgentArena average score — 66.2

• OSWorkerBench strict success — 41.00

• OSWorkerBench progress — 76.86

OSWorkerBench is itself one of the three contributions in the paper: a new benchmark of 100 long-horizon office tasks across 41 applications, with 33 self-demo and 45 variant-demo subsets. It is a new evaluation, so there is no prior art to compare those two scores against. Against its own base model, UI-Mate-27B is claimed to beat Qwen3.6-27B by 17.7 points of strict success and 24.5 points of progress on OSWorkerBench — which is the cleanest apples-to-apples number in the whole release, since both models would run through the same harness.

Every figure above is the team's own evaluation. There are no independent scores yet, no third-party reruns, and with zero downloads, no community reproductions either. Treat every number here as a claim, not a fact.

Where those numbers would sit — if they hold

WindowsAgentArena is the one that would be a genuine headline. The best publicly reported WAA results earlier in 2026 clustered around 61.0 (a modular system called VLAA-GUI, April 2026); the open-weight EvoCUA-32B from Meituan sat at 56.5, and ByteDance's closed UI-TARS-2 was in the low-to-mid 50s on the same board. A 66.2 on UI-Mate-27B's own eval would put it above everything that had been reported on this benchmark this year — open or closed. That is a strong claim, and it needs an independent rerun.

OSWorld-Verified at 77.0 is strong but not a new open-weight record on the public board. On the OSWorld-Verified leaderboard snapshot from early August 2026, the open-weight Holo3-35B-A3B is listed at 82.6, with several frontier closed models above it (Qwen3.8 Max at 86.1, Claude Mythos 5 and Claude Fable 5 at 85.0, Claude Opus 4.8 at 83.4). The paper's "state of the art" wording is therefore a claim about its own evaluation setup, not a settled fact — different harnesses, action parsers, and self-reporting drift all make 77.0 versus 82.6 a question only an independent rerun can settle. For context on what a 27B open model doing 77.0 means: it would sit above the ~72.4 human-expert baseline and above several larger frontier systems on that same board, including Claude Opus 4.6 at 72.7 and Kimi K2.6 at 73.1.

Tencent is not new to this territory — it shipped POINTS-GUI-G, an 8B GUI-grounding model, in February 2026, launched the Marvis OS assistant in May, and contributed to the GUICrafter computer-use project in June. UI-Mate is a distinct line aimed specifically at full desktop control, and it is the first of Tencent's GUI agents to ship as an open foundation model rather than a grounding component or a product.

How to run it

The model is designed for vLLM, and the model card gives the exact command — vllm serve tencent/UI-Mate-27B with tensor-parallel size 2 and the OpenAI-compatible chat template. One practical detail stands out: the agent retains five screenshots in context by default, so the server must admit at least six images per prompt because the newest screenshot arrives before the oldest is collapsed. The recommended flag is --limit-mm-per-prompt with six images and zero video. The demonstration-guided checkpoint serves the same way — its card names vLLM and SGLang behind an OpenAI-compatible endpoint.

Once served, a Python agent class (UIMateAgent) takes a screenshot and an instruction and returns the response plus structured actions, rescaling the 1000×1000 coordinates back to your resolution. The project page also offers a macOS Apple Silicon app for the demonstration-guided mode, which is the easiest way to try the "show it once" workflow without building the harness yourself. Licensing is Apache-2.0 throughout, so local use, fine-tuning, and commercial integration are all allowed.

Two warnings belong next to any "how to run it" for a computer-use agent, and both come from the model card itself. Use isolated or disposable environments. Require human confirmation before anything sensitive, monitor the trajectory, and do not treat a model-reported success as proof that the intended outcome actually happened. GUI agents misclick, and prompt injection through on-screen content is a real threat model, not a hypothetical.

The stack around a new GUI agent

None of the UI-Mate checkpoints are on OrcaRouter yet — the family is days old with zero downloads, and its base model Qwen3.6-27B is not either. There is no hosted API, so if you want to run one this week, you are serving vLLM yourself. That is fine for a lab, but it is the wrong shape for production.

A production computer-use pipeline is rarely one checkpoint. It is a planner model deciding the next intent, a grounding or vision model reading the screen, the GUI agent itself, and a cheap fallback when a sub-step fails — and those pieces are all served models, even when the GUI agent is local. That mix is exactly what a routing layer is for: one API across 200+ hosted models, provider list price passed through at 0% markup, and automatic failover so a transient outage on one provider does not stall a long agent run. The routing DSL also lets you compose several models into a single call — a planner up front, a cheap verifier at the end — without stitching providers together in your own code. The honest summary is simple: the agent that drives the desktop is local, but the models that decide what to do around it are routable, and trying brand-new unproven checkpoints safely is precisely what failover is for.

What to watch next

Four things would change the picture for UI-Mate-27B, in rough order of importance:

• An independent rerun of OSWorld-Verified and WindowsAgentArena. The WAA figure in particular is either a big deal or a harness artifact, and only an outside eval settles which.

• The formal technical report. The current citation is a placeholder, and the full paper will presumably surface the data-engine details the abstract only sketches.

• Tencent's own announcement. A repo-first drop like this usually precedes something — a Marvis integration, a hosted offering, or a broader open-model push.

• A hosted API. All three checkpoints' weights are public now, but nothing is served anywhere — no Tencent endpoint, no third-party inference provider — so self-hosting remains the only route, and only a Tencent announcement or a provider pickup changes that.

FAQ

What is Tencent UI-Mate-27B?

UI-Mate-27B is an Apache-2.0, 27-billion-parameter foundation GUI agent from Tencent's HY Frontier Multimodal Agent Team, fine-tuned from Qwen3.6-27B. It watches live desktop screenshots, reasons over them, and outputs pyautogui-compatible mouse and keyboard actions. It was quietly released on Hugging Face on August 14, 2026 — alongside the 9B sibling and the demonstration-guided UI-Mate-democua-27B — with no announcement, and its benchmarks are entirely vendor-reported so far.

How is demonstration-guided execution different from replaying a script?

Instead of executing a recorded macro, UI-Mate treats a demonstration as a captioned, subtask-structured workflow injected as guidance. The live screenshot stays authoritative, so when the app layout, content, or state differs from what was shown, the model re-plans rather than replaying stale coordinates. That is the mechanism behind the claimed jump from 17.2 to 35.4 strict success on the self-demo subset — and it is still a vendor claim until someone reproduces it.

Is UI-Mate-27B really state of the art for open-weight GUI agents?

That depends entirely on evaluation setup. Its WindowsAgentArena 66.2 would top the best publicly reported WAA results from earlier in 2026, but its OSWorld-Verified 77.0 sits below the open-weight Holo3-35B-A3B at 82.6 on the public board. The paper calls it a new open-weight state of the art; an independent rerun on a consistent harness is the only way to know.

Can I run UI-Mate-27B today?

Yes, if you serve it yourself: download the weights from Hugging Face — the 27B general checkpoint, the 9B, or the demonstration-guided UI-Mate-democua-27B — run the vLLM command from the model card (tensor-parallel size 2, six images per prompt), and drive it with the Python agent or the macOS app. There is no hosted API, and none of the checkpoints are on OrcaRouter yet — which, for models this new and this unverified, is a reasonable reason to keep them in an isolated environment for now.

The right read on UI-Mate-27B is not "the winner" but "worth watching." A 27B open model claiming a WAA lead and shipping a one-shot demonstration workflow is a meaningful datapoint in a year where open-weight computer-use agents have gone from a joke to within reach of the frontier. Now that the demonstration-guided checkpoint is public weights rather than a project-page placeholder, the "show it once" workflow is something you can actually run. Tencent has been careful about releases before, and this one has the shape of a work-in-progress made public. Run it in a sandbox, rerun the benchmarks, and keep the announcement watch going — the interesting part of this story is still ahead.