Jev and Laya battle on latency, cost, and high-cardinality classification

A comparative analysis of Jev and Laya, two pioneering non-generative AI decision models designed to replace slow, expensive LLMs in classification, scoring, and boolean evaluation pipelines.

Eliminating the generation tax

Both Jev and Laya operate on a simple premise: production pipelines often waste compute on text generation when they only need a structured label. Instead of waiting for a large language model to stream tokens that must then be parsed, these decision models accept an input state and immediately return a typed answer with an associated probability. This architecture eliminates hallucinations by ensuring the output always conforms to a predefined schema.

Three core decision types

The interfaces of both models are built around three specific query types. The Choice interface selects a single option from a closed list and returns a probability distribution across all options. The Score interface places the input on an ordered scale, such as relevance from 0 to 5. The Noul interface evaluates the probability that a given statement is true. Both models support batching, allowing multiple independent questions to be processed in a single request.

High cardinality vs raw speed

The architectural divergence between these two models is stark. Jev, a hosted API developed by Diogo Almeida at TypeSafe AI, handles high-cardinality classification tasks with high accuracy. On the Banking77 benchmark, which requires selecting from 77 intent labels, Jev achieved a score of 0.870. Laya, developed by Convai Innovations, struggled on the same benchmark, scoring 0.425. This performance gap is architectural. Laya shares a fixed token budget of 192 to 256 tokens across all candidate options, meaning each option receives minimal representation when the choice list exceeds 20 categories.

However, Laya dominates on latency and deployment flexibility. As a 421M-parameter encoder based on ModernBERT-large, Laya runs locally on your own hardware. Convai Innovations claims a latency of 33 milliseconds per query on a single GPU, which drops to 7 milliseconds per question when batched. This is significantly faster than Jev's reported 150 milliseconds network round trip. Laya also offers a 322M multilingual variant covering over 100 languages and can be fine-tuned in about 4 hours using a free GPU. Jev is closed-source, meaning fine-tuning is not an option, and its expected calibration error of 0.144, while better than Laya's 0.213, must be weighed against the lack of local deployment options.

Pricing

Pricing snapshot as of September 2026:

• Jev (TypeSafe AI): Hosted API priced at $0.042 per million input tokens. Output tokens are free. There is no free tier.
• Laya (Convai Innovations): Open-source under the Apache 2.0 license. Model weights are free to download on Hugging Face. Operational costs are limited to the self-hosted GPU infrastructure required to run the model.

Verdict

For teams routing complex, high-cardinality datasets, Jev is the logical choice. Its Banking77 score of 0.870 and superior expected calibration error of 0.144 make it highly reliable for complex schemas without requiring manual calibration. However, for latency-sensitive applications or strict privacy requirements, Laya is the better option. Its 33ms single-query latency and open-source Apache 2.0 license make it a powerful tool for teams capable of managing their own GPU infrastructure.

What we'd test next

In our next phase of testing, we aim to build a dedicated test rig to verify the latency claims of both models under sustained load. We will benchmark Jev's API latency from different geographic regions to measure actual network overhead. For Laya, we plan to run independent benchmarks on standard consumer GPUs to verify the 33ms single-query and 7ms batched latency claims, and evaluate the 4-hour fine-tuning process on a single free GPU.

The investor read

The shift from generative LLMs to specialized, non-generative decision models represents a critical consolidation of the AI tooling stack. For years, enterprises have overpaid for massive, general-purpose LLMs to perform simple classification tasks. Jev and Laya prove that specialized encoder-based architectures can deliver superior accuracy and latency at a fraction of the cost. From an investment perspective, TypeSafe AI's hosted model (Jev) targets high-margin enterprise workflows that require high-cardinality classification and out-of-the-box calibration. However, the rapid emergence of Laya as an open-source, self-hosted alternative based on ModernBERT-large signals that basic classification is rapidly commoditizing. To remain investable, hosted decision APIs must offer proprietary fine-tuning pipelines, advanced security guarantees, or unique multi-modal capabilities that cannot be easily replicated on-premise with open-source weights.

Pull quote: “Instead of waiting for a large language model to stream tokens that must then be parsed, these decision models accept an input state and immediately return a typed answer with an associated probability.”