GitHub · by Nandakishor M

Laya

Open weights that answer typed questions in one forward pass, 33 ms on a T4.

Akshay 🚀

@akshay_pachaar

A faster, 100% open-source alternative to Jev. It’s called Laya. Both models solve the same problem. You give them unstructured state plus typed questions. Instead of generating a paragraph, they choose from answers you define and return probabilities your code can act on. But they make those decisions differently. Jev is a closed, hosted API. Its architecture and training are not public, but it works zero-shot and supports up to 255 options. Laya is Apache 2.0 licensed, runs locally, and uses open-weight encoder models. Here is how its pipeline works. → It routes by language first. Plain Python selects the English or multilingual checkpoint before inference. → It creates one sequence for each question. Every possible answer gets a `[MASK]` marker. The question and state provide the context needed to compare those options. → It scores instead of generating. A bidirectional encoder and small decision head assign one score to each option. Softmax turns those scores into a probability distribution. → It batches the questions. They run together in one model call, although every question still carries its own copy of the state. This removes network latency and token-by-token generation. In one independent test, Laya took about 35 milliseconds on a local GPU. Jev took roughly 380 milliseconds over the network. Laya also has no per-call API fee, keeps data on your infrastructure, and can run on older GPUs. But it is not a drop-in replacement for every workload. Jev is a hosted generalist that works well out of the box. Laya is a fast base you need to specialize. Laya makes the most sense when you have a stable answer set and representative training data. Fine-tune it for the task, calibrate its probabilities on held-out examples, and send uncertain decisions to a stronger model or human reviewer. Jev gives you stronger zero-shot decisions. Laya gives you speed, privacy, control, and no per-call fee after you teach it your task. The open-source repository is here: http://github.com/NandhaKishorM/laya If you want to self-host Laya, InstaCloud provides a really nice and easy way to do so. Check this out: https://www.instacloud.com/templates/laya If you're new to decision models, I wrote the full breakdown of how Jev. The article is quoted below.

Oct 7, 2026 · 225 likesOpen on X

Nandakishor M’s open decision model, Apache 2.0 and installable with pip. It answers the same three typed primitives Jev takes — choice, score and noul — without generating any text, so there is nothing to parse. Three checkpoints ship together: 421M for English, 322M for 100+ languages and 421M tuned on typed decisions, with a Router that detects the script of the incoming state in under half a millisecond and dispatches to the right one. The repository reports 32.8 ms for a single question on a T4 and 7.2 ms a question in batches of ten.

The design point is the same one every Jev build rests on: the question has a fixed answer set, so the model never has to write. Laya scores the options in a single forward pass of an encoder — ModernBERT-large for English, mmBERT-base for the multilingual checkpoint — and returns a label with a probability, an expected level on an ordinal rubric, or a calibrated P(true). The probabilities are trained against strictly proper scoring rules, which is what the repository leans on when it tells you to branch on the confidence: act automatically above 0.85, escalate below it.

The comparison table is the reason this repository travelled. Against third-party published Jev figures — the author has no TypeSafe API access and says so — it reports 0.766 against 0.727 on 2,000 typed decisions, 0.950 against 0.910 on AG News, 0.595 against 0.480 on DAIR Emotion, and an expected calibration error of 0.081 against 0.246 after temperature fitting. The latency line is the widest gap: 32.8 ms against the 236–276 ms p50 that AbdelStark and nibzard measured for Jev, about seven times faster, on weights you host yourself.

It also publishes where it loses, which is worth more than the wins. On Banking77 Jev scores 0.870 on 72 labels and Laya 0.425 on 77, because the options share a fixed head budget — 192 tokens on English, 256 on multilingual — so 77 labels get three or four tokens each and stop being distinguishable. Jev takes up to 255 options out of the box. Jev also wins on soft distribution matching, 0.580 against 0.471. Over fifty options in one call, Laya needs `head_max_len` raised or the shortlist path.

One deployment note the README puts in bold, because it turns a 33 ms model into a 10 second one: the Router keeps a single checkpoint resident by default, so traffic that alternates languages rebuilds a model on every request — a measured 7.4 s median on CPU and 10.3 s on a T4. Serve it with `Router(preload=True)`. The English checkpoint on its own is not a fallback for other scripts either: Khmer scores 0.000 accuracy at 0.952 confidence, so the model stays confident while being wrong and no confidence gate catches it.