GLM-Legal: Z.ai's Legal Agent Scores 19.2% on Harvey LAB — and Starts as a Beta You Apply For

Z.ai published GLM-Legal on 12 October 2026, and the number it leads with is 19.2%. That is the model's all-pass rate on Harvey LAB, a law-firm workflow evaluation built by the legal-AI vendor Harvey across 24 practice areas — and it is the most trustworthy figure in the release precisely because it looks unimpressive. A legal model that clears fewer than one task in five on a serious harness is being graded against a standard its own vendor did not soften. The weaker part is everything around it: the other two headline benchmarks, LegalBench and PLawBench, are described as parity with unnamed "frontier closed-source models" and carry no numbers at all. GLM-Legal is built on GLM-5.3-Flash, Z.ai's 320B-total, 18B-active multimodal foundation model, and right now the way to get it is a form on the vendor's consultation page, not an API key.

That combination — a real, honestly-graded number plus two unquantified parity claims — is the whole shape of this release, and it is worth separating before anything else.

What 19.2% on Harvey LAB says, and what it does not

Harvey LAB grades on all-pass. A task counts for the model only if every step lands: the right issues identified, the right provisions cited, the analysis defensible, the output formatted to the brief. That is a harsher gate than a partial-credit benchmark, and it is why the absolute number is low rather than why the model is weak. Z.ai frames it as confirmation of "reliable end-to-end task execution without intermediate degradation" — in an all-pass regime, that is the only thing a passing task can mean.

The comparison Z.ai supplies is against its own base model, and it says GLM-Legal improves by 12.5 percentage points. Read that against the headline and the arithmetic is stark:

• GLM-Legal on Harvey LAB — 19.2% all-pass, across 24 practice areas, per Z.ai.

• Implied base-model score — roughly 6.7%, since Z.ai states the gap as 12.5 points and the base is GLM-5.3-Flash.

• Independence — none. Both figures are Z.ai's own, measured on Z.ai's own run, and no third party has reproduced them.

• Harvey's role — Harvey built the benchmark; the announcement does not say whether Harvey ran the evaluation or Z.ai did.

That last line is the honest gap. "Built by Harvey" and "scored by Harvey" are different claims, and only the first is made. A 12.5-point lift over a base model is a plausible order of magnitude for domain mid-training plus RL — this is what legal specialisation looks like when it works — but until someone outside Z.ai re-runs the harness, it is a vendor number, and it should be quoted as one.

The two claims with no number attached

LegalBench covers contract parsing, merger filings and judicial opinions; PLawBench covers Chinese legal consultation, case analysis and document drafting under fine-grained rubrics. Z.ai's claim on the first is that GLM-Legal parses them "with the precision of top closed-source frontier models," and on the second that it "achieves parity with leading frontier systems."

Both claims are qualitative. Neither comes with a score, a confidence interval, or a named comparator. That is not a reason to disbelieve them — GLM-5.3-Flash is a strong base, and a domain-specialised derivative of a strong base does often reach parity on domain reasoning — but a parity claim you cannot put a number on is uncheckable by design, and the two quantitative statements in the release are both about the same benchmark. A reader comparing this launch to a competitor's should notice that the release is really one measured result plus two argued ones.

What the training actually changed

The methodological detail matters more here than in a typical domain fine-tune, because GLM-Legal is a weights-level specialisation rather than a retrieval product. Z.ai did not ship a case-law index; it changed the model. Two stages:

• Mid-training — expanded high-quality legal corpora, teaching statutory rules and interpretive frameworks inside dense domain context, to strengthen the deductive chain from facts to rules to conclusions.

• Post-training — reinforcement learning across what Z.ai calls "tens of thousands of realistic legal environments," where the model learns to plan an analysis, advance a workflow and produce a deliverable under concrete constraints.

The unusual part is the loop around the post-training stage. Z.ai describes the agent harness — the runtime orchestration layer that stages context, connects tooling and routes execution feedback — as co-evolving with the model rather than sitting fixed underneath it. Edge cases that surfaced in training exposed harness bottlenecks; harness refinements exposed the model to more realistic execution constraints. The stated purpose is to stop the classic domain-model failure where knowledge is added but workflow behaviour is not.

Whether that generalises is exactly what 19.2% all-pass is testing. It is a reliability claim, not a knowledge claim, and it is the right thing for a legal deployment to be measured on.

The three demos, read closely

The blog post shows GLM-5.3-Flash and GLM-Legal answering the same three prompts side by side. The comparison is vendor-authored and the prompts are vendor-chosen, so it proves less than a head-to-head would — but the divergences are specific enough to be informative.

Contract review. The task is a software services contract for a client switching providers. The base model's advice is reasonable and general: shorten the notice period, get the provider to return data and paid-for deliverables, add a duty to cooperate with the replacement provider. GLM-Legal's advice is the same shape but resolves the numbers — reduce the convenience-termination notice from 60 to 30 days, remove the cure period entirely for breaches of IP, confidentiality or data-security obligations, require return of work product whether or not it has been paid for, and add up to 90 days of transition assistance at the provider's then-current rates. That is the difference between a checklist and a redline. Z.ai says AutoClaw applies those edits into the Word document as tracked changes and comments.

Evidence analysis. Forty documents from a software project dispute. The base model spots the contradiction between the 30 September client report claiming 60% complete and the internal snapshot at roughly 41%, and correctly says the documents "may support" the client's position. GLM-Legal links the 8 September internal warning that the milestone could not honestly be met, the 41% snapshot and an executive instruction to suppress the risk and keep invoicing into a single concealment theory, then cites the master services agreement's payment-does-not-waive-rights and surviving-warranties clauses as the reason continued payment does not settle the dispute. The base model found the facts; GLM-Legal built the argument and named the claim elements still to test.

Compliance research. A website launching in Singapore, Malaysia and Indonesia with tracking, email marketing and overseas cloud storage. The base model produces a competent pre-launch to-do list. GLM-Legal produces a prioritised one with statutory hooks — SG PDPA s26, MY PDPA s129, ID PDP Law Arts 55–56 — and separates shared controls from market-specific questions.

The statute citations are the part a lawyer can check in an afternoon, and they are why the demos are worth reading at all. The part they do not show is how the model behaves on a matter where the correct answer is that the client is wrong.

Access: a form now, AutoClaw later

There is no published price and no public endpoint. Z.ai's call to action is "Join the GLM-Legal Beta," aimed at law firms, legal-tech companies, in-house teams and independent practitioners, with API access behind an application at the vendor's consultation page. The AutoClaw paragraph is written in the future tense — "you'll soon be able to put these capabilities to work in AutoClaw" — so the desktop agent platform is a coming integration, not a shipping one, and Z.ai does not commit to a date for either.

If you want to build against the underlying model today, the honest split is that the base is available and the legal variant is not. GLM 5.3 Flash is on the OrcaRouter catalogue at Z.ai's list price with no markup added — $0.15 per million input tokens and $0.50 per million output tokens, with cache reads at $0.030 — so a vendor price change on that model shows up on our side the same day rather than at the end of a contract cycle. One key reaches more than 200 models, and automatic failover means you can point a staging path at the base model without betting a production route on it. Everything in the three demos that does not depend on legal post-training — restructuring a clause set you supply, formatting output to a firm template, generating a first-pass review checklist — you can prototype on GLM 5.3 Flash now and swap the model id when the beta opens. Everything that depends on the specialisation you cannot prototype, because the specialisation is the part behind the form.

What would change this story

Three things, in order of how much they matter. A number for LegalBench and PLawBench — the parity claims are the load-bearing ones for anyone choosing a legal model, and they are currently the least checkable. An independent run of Harvey LAB, ideally one where Harvey rather than Z.ai controls the harness. And a price, because a domain model that costs meaningfully more per matter than its base has to earn that difference on the all-pass rate rather than on polish.

None of the three has arrived. What has arrived is a genuinely specialised model built on an 18B-active foundation, with one benchmark run under a grading standard strict enough that 19.2% reads as credible, and an access process that says the vendor is not yet confident enough to put it on a rate card.

For a legal team deciding what to do with this week's announcement, the answer is probably the same as with any application-gated release: read the demos for the failure modes they reveal, check the statute citations yourself, and build the surrounding pipeline on a model you can actually call today so that swapping one model id is the entire migration.

automatic failover means you can point a staging path at the base model without betting a production route on it.