GitHub · by Nandakishor M

Laya

Open weights that answer typed questions in one forward pass, 33 ms on a T4.

Nandakishor M’s open decision model, Apache 2.0 and installable with pip. It answers the same three typed primitives Jev takes — choice, score and noul — without generating any text, so there is nothing to parse. Three checkpoints ship together: 421M for English, 322M for 100+ languages and 421M tuned on typed decisions, with a Router that detects the script of the incoming state in under half a millisecond and dispatches to the right one. The repository reports 32.8 ms for a single question on a T4 and 7.2 ms a question in batches of ten.

The design point is the same one every Jev build rests on: the question has a fixed answer set, so the model never has to write. Laya scores the options in a single forward pass of an encoder — ModernBERT-large for English, mmBERT-base for the multilingual checkpoint — and returns a label with a probability, an expected level on an ordinal rubric, or a calibrated P(true). The probabilities are trained against strictly proper scoring rules, which is what the repository leans on when it tells you to branch on the confidence: act automatically above 0.85, escalate below it.

The comparison table is the reason this repository travelled. Against third-party published Jev figures — the author has no TypeSafe API access and says so — it reports 0.766 against 0.727 on 2,000 typed decisions, 0.950 against 0.910 on AG News, 0.595 against 0.480 on DAIR Emotion, and an expected calibration error of 0.081 against 0.246 after temperature fitting. The latency line is the widest gap: 32.8 ms against the 236–276 ms p50 that AbdelStark and nibzard measured for Jev, about seven times faster, on weights you host yourself.

It also publishes where it loses, which is worth more than the wins. On Banking77 Jev scores 0.870 on 72 labels and Laya 0.425 on 77, because the options share a fixed head budget — 192 tokens on English, 256 on multilingual — so 77 labels get three or four tokens each and stop being distinguishable. Jev takes up to 255 options out of the box. Jev also wins on soft distribution matching, 0.580 against 0.471. Over fifty options in one call, Laya needs `head_max_len` raised or the shortlist path.

One deployment note the README puts in bold, because it turns a 33 ms model into a 10 second one: the Router keeps a single checkpoint resident by default, so traffic that alternates languages rebuilds a model on every request — a measured 7.4 s median on CPU and 10.3 s on a T4. Serve it with `Router(preload=True)`. The English checkpoint on its own is not a fallback for other scripts either: Khmer scores 0.000 accuracy at 0.952 confidence, so the model stays confident while being wrong and no confidence gate catches it.