Can you fine-tune Jev?

Why you cannot fine-tune Jev itself

Fine-tuning needs weights to start from. TypeSafe serves Jev over an API and has not published its weights, so every account calls the same model, and there is no checkpoint to adapt. Nothing on this page changes that: what follows is how people get a model that answers the same typed questions their own way.

The usual reason to want one is drift — the model keeps getting a particular case wrong and you have the corrected examples to prove it. Jev takes that correction through the question instead of through the weights: the state you pass, the options you list and the criteria you write are the whole surface you control. Whether that is enough is the question the rest of this page answers with other people’s numbers.

Fine-tuning a Jev replacement on Qwen3.5 4B for $17

Together AI’s fine-tuning walk-through (the long-form version is on their blog) trains a classifier on top of Qwen3.5 4B. It samples about 38,000 questions from eight public datasets — MultiNLI, BoolQ, Banking77, AG News, SST-5, plus three synthetic sets for policies, routing and paper classification — normalises them into one format, and launches a fine-tune on Together’s service. The training job takes roughly 25 minutes.

The result is a model that takes a state and a set of labelled options and returns the letter of the best one. In the worked example, a customer message about a duplicate charge comes back as { "label": "A", "key": "duplicate_charge" }. It runs on a dedicated endpoint, and Together’s hosted version is priced at $0.042 per million input tokens with output free — the same price as Jev. The code is at togethercomputer/tev1.

What the $17 fine-tune cost leaves out

The $17 is honest and it is narrow. Four things sit outside it, and each one is where a fine-tune project actually spends its time.

The data is the work. The run samples, normalises and formats eight datasets into one question-answer shape. When you do this for your own schema, that collection and normalisation is the job, and it is not in the $17.

The endpoint is a separate line. A fine-tuned model is not useful until it is served. Together’s own steps deploy it to a dedicated endpoint on an H100, which is metered apart from the training.

The schema still has to be written by hand. The datasets are a generic spread of classification tasks. The value in your model is your categories and your questions, and those are the part no dataset supplies.

Retraining is recurring. The moment your categories change, the model is answering a question it was not trained on. A fine-tune is a snapshot; a decision system that is still being discovered is not ready to be frozen.

One detail from the article is easy to miss and matters when you serve it yourself: the public endpoint does not inject the inference settings, so a direct call has to set temperature=0, max_tokens=8 and chat_template_kwargs={"enable_thinking": false} explicitly.

Three ways to a Jev-style decision model (only one is fine-tuning)

7Stars

GitHub·Open source

open-jev (PyTorch)

A from-first-principles rebuild of the ideas behind Jev.

KGKye Gomez

Vincent Wang-Maścianica

@vinnylarouge

I reverse-engineered a jev-like architecture given its type. You can find the repo here to train your own jevlikes: github.com/vinnylarouge/j…

GitHub·Open source

jevlike

10Stars

GitHub·Open source

JevTuner

Tuning for Jev-style decisions.

Lliushiliushi

Geek Lite

@QingQ77

复刻 Jev 的并行决策思路，让 0.6B 小模型一次前向直接输出完整概率分布，不做输出 token 解码。 github.com/TianyuCodings/…

GitHub·Open source

NanoJev

Size

0.6B

Adapt an existing model. QwenJev converts Qwen-3.5 4B to answer like Jev and reports 0.169 seconds a frame. NanoJev copies the parallel-decision approach in a 0.6B model, emitting the whole distribution in one forward pass. Laya is the strongest open comparison, and its own table is the most honest document in the field.

GitHub·Open source

Pick

Laya

Open weights that answer typed questions in one forward pass, 33 ms on a T4.

NMNandakishor M

13.7k

GitHub·Open source

kev

A tiny Jev-like model on Qwen2.5-0.5B that trains and runs on a MacBook.

JPJared Palmer

3k~160 ms

0.169 sPer frame

GitHub·Open source

QwenJev

A Jev-style conversion of Qwen-3.5-4B, at 0.169 s per frame.

RRJMSWD

12

74Stars

GitHub·Open source

OpenJev

An open attempt at a Jev-class decision model.

SSiliconLabAI

Nick Khami

@skeptrune

you can make any open source model behave like jev with just a bit of inference engineering. it's shockingly easy. to prove it, we built a new endpoint we're calling deepseek-v4.1-flash-jev. see the demo below. here's how it's done: sglang (an inference engine) offers a scoring endpoint in addition to the normal generation one. in scoring mode, given an input & set of possible answers, it forces the model to produce probabilities for each one. example: > input: what is most common letter in abcccde? > possible answers: a, b, c > output: (c, 0.9), (b, 0.0.5), (a, 0.05) getting the above behavior instead of streamed output is as simple as using sglang's /v1/score endpoint instead of /generate. there's just one other trick required. for deepseek, you have to add a closing think tag before the response. this forces a direct answer instead of a reasoning trace. if you want reasoning, you can do that too, but imo that makes things too slow to be worth it. dsv4.1 flash is not as good as jev, but if we had enough spare compute to experiment with this same approach for a larger model then i think the decision quality would be at least as good, if not better. also, somewhat unrelated, i think decision-making models kill all prospecting & sourcing work. i would have absolutely killed to have jev or similar when i was recruiting @mintlify. absolutely incredible.

X·Open source

deepseek-v4.1-flash-jev

Train nothing. Restrict the next token to your options, read the candidate logits, and you have a typed decision with probabilities out of a model you already serve. Simple Jev turns any Hugging Face model into that endpoint; AnyJev ships the same premise as a library from Nokia’s applied research group.

Eugene Cheah - AI builder @ 🇸🇬|🇺🇸

@picocreator

love jev, but upset it - isn't open source? - it lack vision capability? We fixed all of that, introducing SimpleJev.ai A fully open source library which takes any HF model and Jev-ify it, with an API endpoint Now on github, and live in production at @FeatherlessAI

GitHub·Open source

Simple Jev

12Stars

GitHub·Open source

AnyJev

Turn any LLM into a Jev-style decision model, with no training.

NANokia Applied Research

Anand Prasad

@theanandprasad

you can turn any low-cost open-source LLM into a Jev like fast decision model without training it. Here's how to do it: An LLM doesn't write text directly. At every step it scores every word in its vocabulary for how likely that word is to come next, and then it picks one. Those scores are called logits. Suppose you want to know whether a customer email is angry. The usual way is to ask the model, wait while it writes a paragraph, and then try to pull a yes or no out of the response. The trick is to never let it write anything. Give it the email content and the question, and end your prompt with "Answer:". Your full prompt will look something like this: [ Message: "Third time I've contacted you. Still broken." Is this customer angry? Answer yes or no only. Answer: ] The model reads the prompt once and scores every possible next word. You look at two of those scores. "yes" scores 8.1 "no" scores 5.9 Convert those two scores into probabilities and you get 90% yes and 10% no. (Use softmax function -- google it) Your code then uses that number directly. If it's above 0.8, escalate the ticket. Congrats, you have created your own Jev! Here’s why it works: 1. It's fast. Reading a prompt happens in one parallel pass. Writing is the slow part because the model writes one word at a time. Here the model never writes. 2. It's cheap. Output tokens are the expensive ones, and this produces none. 3. There's nothing to parse. You always get a fixed output, and never a paragraph. 4. You get a confidence level along with the answer. A 0.55 means the case should go to a human or a bigger model. A 0.98 means you can act on it. I tried running the code on llama 3.3 70b (via openrouter) and the accuracy was 21/22 (95%). Cost per query: $0.000013, latency: ~500ms

X·Open source

Jev-like decisions from open LLMs

Petru - Tech Driven

@techdrivenpetru

Yes but not with Jev. I used another classifier, locally, just as fast, just as good. Example: setup a "server" in python that loads the classifier model. Add a hook in Claude that fires on "pre-tool-use" and next time you ask Claude a random question like "how do I lint check a project?" and it tries to freelance and read your entire repo only to intoxicate itself and pollute its context, the classifier will slap its hand, say "no sir, you answer from knowledge" and deny the tool call. I tested this yesterday with success, but need to refine it as it misfires. Basically I was able to identify general queries, instances where I would ask something and Claude would rush ahead and run pip install without me asking or just write code instead of answering. A classifier is hypercheap compared to a regular LLM and would catch all of these. The model I used was DeBERT large. It's still stupid fast, I tested it on an Apple with M1 (regular) and you don't feel it running.

X·Open source

A local classifier instead of Jev

さっと

@satto_sann

Jevでサクッと分類器は価値があるのかビルドトラップできそう

X·Benchmarks and evals

Is a quick classifier worth building?

Laya vs Jev: where the open fine-tune wins and where it collapses

The reason Laya is worth reading is that it tests itself against third-party published Jev figures and then says where it loses. It is the only document in this category that does both.

That last row is the lesson. Laya’s options share a fixed token head — 192 tokens on English, 256 on multilingual — so 77 labels get three or four tokens each and stop being distinguishable. Jev takes up to 255 options in a choice. And a local router that keeps one checkpoint resident will rebuild a model on every language switch: a measured 7.4 seconds on CPU and 10.3 on a T4, which turns a 33 ms model into a ten-second one.

When a fine-tune beats Jev, measured

The most useful comparison published so far is distil labs’ two-step accounts-payable pipeline, because it runs Jev, hosted LLMs and fine-tuned Qwen3.5 models through the same two jobs and reports both. It is also the one result that argues against the rest of this page, which is why it is here.

The split is the lesson, and it is sharper than “train or don’t”. Where the answer can be read off the input, Jev ties a fine-tune at full marks and needs no training run at all. Where the step has to weigh several facts against a policy before answering, their fine-tune reached 0.98 and Jev averaged 0.79. Sort your decisions by that line before you price anything: the readable ones never justify a training budget, and the ones that need judgement are the only place a fine-tune has ever paid for itself in public.

Should you fine-tune, or just call Jev? A four-question check

The directory has two builds whose whole subject is whether this is worth doing: the traps of building a classifier fast and someone who skipped Jev for a local classifier and got the same result for their case. Both point the same way: calibrate against labels you already have before you spend the training budget.

1. Can you write the acceptable answers down, and will they hold for months? If not, a frozen fine-tune is the wrong tool.
2. Can you get the same result by constraining an open model’s logits, with no training? Try that first; it is reversible in an afternoon.
3. Do you have a few hundred cases with known answers to calibrate on? Without them you cannot set a threshold, and without a threshold the model is a demo.
4. Does the open model publish where it loses? If it does not, assume there is a high-cardinality or long-tail case it has not met.

For the ready-made options rather than training your own, read the open-source Jev page; for whether a local model is fast enough, Jev vs an LLM; for what a hosted call costs instead, the pricing page. The decision-model papers and independent evaluations are collected on papers and open reproductions.

Where these numbers come from

Every figure above is its author’s. These are the write-ups they published, in the order that makes the argument: the $17 run itself, then the people who measured an open model against it.

Hassan

@nutlope

x.com/i/article/2102…

Guide·x.com

How to train your own Jev for $17

Together AI fine-tunes a Jev-like classifier on Qwen3.5 4B: about 38,000 questions sampled from eight public datasets, roughly 25 minutes of training, and the tev1 repository that runs it. The $17 is the training job, not the system around it.

Scott Williams

@swill1ams

Prediction: millionaires will be made using custom Jev style models (parallel constrained decoding) to make the agent systems companies already run more token efficient. Let me explain with a scenario: Imagine a company already has an agent workflow running where an llm reviews every item before it moves on: a support ticket gets triaged, an invoice gets approved or held, a claim gets flagged. Every one of those goes through a frontier model today, a few seconds and a few cents each, on the way to a decision that in most cases is obvious. Behind that flow sits years of humans (or agents) making the exact same call, with the outcome attached. Now imagine you first run each item through a custom PCD or similar model that costs a fraction of the llm and returns a classification of what to do at that step, with a mathematically accurate probability attached. When it's confident, the item skips the llm entirely. When it isn't, the llm handles it as normal. The model has seen years of your team making this exact decision, usually a constrained set of decisions, so it should be right most of the time. Say it comes back confident on 6 out of 10 items. That's more than half your llm spend potentially gone from that step, likely with comparable accuracy. This pre processing idea works in a bunch of other use cases too, such as: - model/request routing: cheap model, frontier model, or a human - picking which skill or subagent to load for a turn instead of stuffing the whole catalog into context - reranking retrieved context so only the relevant chunks reach the window - guardrails on every agent turn: contradictions, policy issues, prompt injection - extracting typed fields from unstructured data emails, PDFs and transcripts before anything expensive touches them Every one of those is a decision an llm makes today, that could potentially be done by another, cheaper model class. Very excited to see Jev/PCD-based pre processing use cases get deployed to agents at scale.

Guide·x.com

Custom Jev-style models for agent workflows

Scott Williams on using parallel constrained decoding to make existing agent systems use fewer tokens.

Guide·astgl.com

Does an open-weight decision model beat a hosted one?

ASTGL compares local Laya with hosted Jev on typed decisions.

Guide·blocks.ai

We rebuilt Jev’s API on an open model and played Doom

Blocks.ai rebuilt Jev’s API on top of an open model and used it to play Doom.

Guide·youtube.com/@AdamGardner

Jev vs. Laya: what they are and the controversy

Adam Gardner on both decision models and the argument around them.

Guide·github.com

Papers and open reproductions

OmniJev’s list of the papers, open reproductions and independent evaluations behind System One models.

Common questions

Can I fine-tune my own Jev?

You can train a model that behaves like it for one schema, and Together AI published a run that does it on Qwen3.5 4B for about $17 in training cost. You cannot fine-tune Jev itself: TypeSafe serves it from an API and has not released the weights.

Is the $17 the whole cost?

No. It is the training cost of the fine-tuning job. It does not include building and normalising the data, the dedicated endpoint that serves the model, or retraining when the schema changes. The article is clear that $17 buys the training, not the system.

Is a fine-tuned classifier the same as Jev?

No. The published run answers one question and returns a single letter. Jev answers many questions about the same state in one pass and returns a probability distribution and a confidence for each. A single-label classifier is a smaller object.

Do I even need to train anything?

Often not. Restricting the next token to your options and reading the candidate logits turns an open model you already serve into a typed decision endpoint with no training at all. Several builds here do exactly that.

How long does fine-tuning a Jev-style model take?

The Together AI run trains in roughly 25 minutes on their service. That is the job, not the project: sampling and normalising about 38,000 questions from eight datasets comes first, and deploying the result to an endpoint comes after.

How much VRAM do you need to fine-tune one?

None, on the published route: the Together run trains on their hosted service and deploys to a dedicated H100 endpoint, so no local GPU is involved. If you want it local instead, kev is a LoRA adapter and a small head on Qwen2.5-0.5B that Jared Palmer trains and runs on a MacBook.

What is the cheapest open decision model?

Laya is the smallest with a published comparison: 421M parameters for English, 32.8 ms on a T4, answering the same choice, score and noul primitives. It also publishes where it loses, which is more useful than the wins.