Jev vs Laya: 33 Milliseconds on a T4, and a Random Baseline

Laya's model card contains the sentence that decides this comparison, and it was written by the people who made Laya. "Laya is a fast base to specialise, not a zero-shot decision engine." Convai Innovations released Laya on September 18, 2026, three days after TypeSafe AI launched Jev, under Apache 2.0, with weights on Hugging Face and a pip install laya entry point. It answers typed questions in a single forward pass with no token-by-token decoding, which is the same architectural bet Jev makes. The headline claim is a p50 latency of 32.8 milliseconds per decision on a Tesla T4, framed as roughly 7.8 times faster than Jev's published 236–276ms p50 through its hosted API. The claim is real and it is also measuring two different things — a local GPU against a network call — and the accuracy picture underneath it is the part that should determine whether you download it. Zero-shot, Laya scores 0.362 on TypeSafe's typed-decisions benchmark. Random guessing scores 0.318. The majority-class baseline scores 0.461. Laya, out of the box, is closer to random than to the trivial baseline.

What Laya actually is

Laya ships as two checkpoints behind a built-in router that dispatches each input to the right one. The English checkpoint is ModernBERT-large at 421M parameters with a 512-token context. The multilingual variant is mmBERT-base at 322M parameters with a 1,024-token window across 100+ languages. Both are non-autoregressive: a single forward pass returns the answer, so there is no decoding loop, no JSON to parse, and no space in which to emit text outside the declared type. That last property is the same structural guarantee Jev offers, arrived at from a different direction, and it is genuinely valuable for anyone who has written retry logic around a model that occasionally forgets to close a brace.

Jev is the closed counterpart: TypeSafe AI's first System One model, launched September 15, 2026, hosted and early-access, with three typed primitives — Choice from a list you supply, Score on an ordered rubric, and Noul, a yes/no claim with a calibrated probability. All questions are evaluated in parallel against one shared read of the state, output is free because there are no output tokens, and input is $0.042 per million tokens. Its architecture, parameter count, and training compute are undisclosed, and TypeSafe has said the details are being kept close to the chest with a paper possibly later.

The latency claim, measured properly

Laya's 32.8ms p50 on a T4 is a real number and a good one. The 7.8x comparison against Jev's 236–276ms is where care is needed, and Laya's own card is unusually honest about why: the Jev figures are third-party published numbers that Convai never measured itself, and the comparison pits a local GPU forward pass against a hosted API call that includes network round-trip and queueing. Strip the network out and the architectural comparison is between two single-pass models, one at 421M parameters and one at an undisclosed size that TypeSafe has never published.

There is also a batching number worth knowing: at a batch of ten, Laya reports 7.2ms per question. That is the shape of the product. Laya is built to be run locally at volume, and on-device community ports such as laya-mlx extend that to Apple Silicon. Viral claims of "50x faster than Jev" are not supported by the project's own published benchmarks, and the honest framing is a large local-versus-cloud gap that is partly architectural and partly just the difference between your GPU and someone else's API.

What the accuracy numbers say, in order

This is where the comparison stops being flattering, and it is worth laying out in sequence because the ordering matters.

• Zero-shot on TypeSafe's typed-decisions benchmark — Laya 0.362, random 0.318, majority-class 0.461, Jev 0.727. Laya is above chance and below the trivial baseline.

• Fine-tuned on the benchmark's own training split — Laya 0.766 against Jev's 0.727. Laya wins, and the win comes from a checkpoint that has seen the benchmark's training data, which makes it a statement about fine-tuning, not about the base model.

• Banking77 intent classification, where the option set is large — Laya 0.425 against Jev's 0.870. Laya degrades sharply past roughly 20 options, and Jev's Choice fields are documented to handle up to 255 before the two-stage scoring pattern is needed.

• Calibration — Laya ships with a mean expected calibration error of 0.466, improving to 0.081 only after per-question-type temperature refitting. Jev is trained with a method TypeSafe calls RLCD, Reinforcement Learning for Calibrated Decisions, and ships every answer with a probability distribution — though TypeSafe also tells users to validate thresholds on their own labelled data, and one independent audit found Jev's calibration varying sharply by task.

The pattern is unambiguous. Laya is a strong base to fine-tune and a weak zero-shot decision engine, and it says so itself. Jev is a strong zero-shot decision engine whose calibration still has to be checked per deployment. Those are different products with different price structures, and choosing between them is mostly a question of whether you have labelled data and a training loop.

The cost arithmetic is not the same shape as Jev's

Jev is $0.042 per million input tokens with output free, which is $42 per billion, and a maximum-size call at the documented ~32,000-token request budget costs well under a cent. Every's independent test put 777 judgments across 37 documents at roughly a quarter of a cent.

Laya's per-call cost is zero at the margin and non-zero in aggregate, and the aggregate is not small. A 421M-parameter model on a T4 is cheap to run and still costs you a GPU, and the fine-tuning step that makes Laya competitive is where the real spend sits — the data labelling, the training run, the evaluation harness, and the per-question-type temperature refitting that takes its calibration error from 0.466 to 0.081. For a fixed taxonomy that never changes, that is a one-time cost that pays back at volume. For a taxonomy that drifts, it is a recurring one, and Jev's zero-shot baseline of 0.727 becomes a much better deal.

There is a third cost that is easy to miss: Laya's English checkpoint has a 512-token context window. That is not a decision model with a small context budget — it is a decision model sized for a paragraph. Jev's roughly 32,000-token request budget is sixty times larger, and even that is an order of magnitude below the generative models both are priced against. If your state is a support thread rather than a support message, neither model's window is the constraint you should be optimising against.

The deployment question is the real difference

Laya's strongest argument is not latency. It is that Apache 2.0 weights on Hugging Face mean the decision never leaves your infrastructure and the endpoint never has a rate limit. For a routing decision made on a medical record, a legal document, or a customer's account history, that is not a preference — it is the deciding factor, and no hosted endpoint at any price wins that argument. TypeSafe's Jev is early-access and waitlisted, and its own documentation notes that rate limits may change without notice.

The counter-argument is what you give up. Jev's undisclosed larger architecture is doing the work that Laya's 421M parameters cannot: the zero-shot gap of 0.727 against 0.362 and the Banking77 gap of 0.870 against 0.425 are both measures of how much knowledge sits inside the model before you train it. Laya's answer is that you supply that knowledge yourself through fine-tuning, and on a narrow fixed domain that answer works — the 0.766 fine-tuned result is proof.

Where the decision layer sits in a real stack

Neither model generates text, so neither is the whole of any workflow. The pattern both are designed for is two models: a decision layer that makes the typed call, and a generative model that handles the part needing prose, code, or an explanation. OrcaRouter does not serve Jev or Laya — Jev is TypeSafe's early-access endpoint and Laya is weights you run yourself — but the generative half of that pattern is exactly what we cover: 200+ models behind one OpenAI-compatible key at provider list price passed through with 0% markup, so a vendor price change is live on our side the same day. The two-model architecture is testable without a second vendor contract, and with automatic failover the generative path does not become a single point of failure while you decide whether the cheap decision layer is calibrated well enough to automate against.

The verdict

If you have a fixed, narrow taxonomy, labelled examples, and a privacy or latency requirement that rules out a hosted API, Laya is the more defensible choice and the fine-tuned numbers support it. If you need the decision to work out of the box, if your option sets run past twenty categories, or if the state is longer than a paragraph, Laya's own model card tells you it is not the product for that job — and the zero-shot score of 0.362 against a majority baseline of 0.461 is the number to keep in view when someone quotes the 7.8x latency figure at you.

What the two have in common is more useful than what separates them. Both eliminate the class of error that comes from output formatting and do nothing about the class that comes from judgment. Both ship probabilities, and neither has a published reliability diagram you can trust without checking. Test the calibration on your own labelled cases before you automate against either — the latency is already commoditised and the confidence value is the part that has to be earned per deployment.