Jev benchmarks, evals and calibration

OpenRouter

@OpenRouter

1/ Jev, a decision model by @typesafeai, sparked a burst of projects and discussion. We tested it using Ori Eval against popular LLMs on OpenRouter at judging. Jev was >5x faster than the next fastest model, and even its slowest requests beat every other model's median.

idan levin

@0xidanlevin

We just ran Jev on our WebMCP benchmark. The result: basically broke the benchmark. Jev + Mercury 2.5 (a fast, low-cost LLM) using WebMCP solved 100% of the tasks at roughly 112× lower model cost than GPT-6 Astra using computer use with code execution. Compared to Astra using screenshot-based computer use, the model cost was 245× lower (!). We also compared Jev operating the browser with and without WebMCP. We used Browser Use’s open-source Ultrafast, with some improvements to the harness to make it more reliable across the benchmark. Jev’s browser-control accuracy on its own was not amazing - adding WebMCP nearly doubled the number of solved tasks, from 25/49 to 49/49, while reducing model cost by 18% (more on why below). The benchmark and methodology are fully open and reproducible. Full results: https://webmcp.com/benchmark A few words on how the Jev + WebMCP harness works and why this is exciting: Jev receives text as input and a set of discrete options it can choose from. With WebMCP, those options are the tools exposed by the website. At each step, Jev sees the task, the available tools and previous results, then picks what to do next. The limitation is that Jev can’t generate arbitrary text, which you need for tool arguments. For example, it can choose the search_products tool, but it can’t generate the search query itself. So we split the work: Jev picks the tool and Mercury 2.5 generates the arguments if needed. This works well because turns out most of the cognitive load in these tasks is around choosing the right action. The argument generation itself is relatively simple, so we can delegate to a small and very fast model. We used Mercury, which outputs 1,000+ tokens/sec and is very cheap. The result is a pretty simple combination: Jev for tool selection + Mercury for arguments + WebMCP for the interface. It ends up being very reliable, very fast, and very cheap. A few words about Ultrafast and why do we think it underperforms: Without WebMCP, Jev chooses from the page’s controls: which button to click, which field to fill, or which option to select. But choosing a valid button is different from choosing the right next step. The agent still has to navigate menus, understand forms, recover from errors and recognize when the task is actually complete. Our hypothesis is that WebMCP makes the decision space much simpler. Instead of figuring out a sequence of clicks through a website, Jev chooses explicit actions that directly advance the task. @typesafeai itself documents weaker accuracy on questions requiring multiple reasoning steps. WebMCP moves much of that complexity into the website’s tools, leaving Jev with clearer decisions and fewer opportunities to go wrong (in a sense WebMCP "compresses" a sequence of clicks into one tool call). Our modified Ultrafast setup solved 25/49 tasks - that is a result for our particular implementation and benchmark, not a universal limit on Jev or Browser Use. We are open to more harness optimization to get this result to perform better, feel free to directly contribute to the benchmark here: https://github.com/nekuda-ai/WindTunnel Browser-use ultrafast: https://github.com/browser-use/jev-ultrafast

Dan Shipper

@danshipper

we almost never test new foundation models but we've been testing this for ~a week @every and it's pretty wild. the kind of things that will be obviously indispensible in 6-12 months it doesn't produce words as output, it produces probabilities. so it can efficiently act as a judge in cases where you'd need a Fable-level model—but in our testing was 25x faster and 600x lower priced excellent vibe check by @hammer_mt on @every: https://every.to/also-true-for-humans/mini-vibe-check-typesafe-s-jev-judged-everything-i-ve-written-in-0-7-seconds?utm_cta_source=home_main_a_3

nader dabit

@dabit3

Also have been playing with @typesafeai Jev, insane! So many immediate use cases and new apps are possible. What a time to be a builder! Sharing some experiments here starting with: Keystroke oracle / predictive launcher: Your launcher ranks by aliases, fuzzy match, and habit. Jev reads intent: type "the pdf I just downloaded" and the newest PDF is already the top hit with a full confidence on every keystroke, in ~100 ms

ILIAS ISM

@illyism

Now using @typesafeai Jev in http://aiseotracker.com, http://linkdr.com, http://genppt.com, etc AI ends up vibe coding so much AI regex slop if you don't read the code, so I can finally move all this hard-coding to Jev and it's insanely fast! Also for regular LLM calls, it is around 10x faster, 50% cheaper

Wayne Sutton

@waynesutton

Ask Jev anything. Give it a try at askjev.ai It won't answer. It will judge. Let's see if we can get to 1 million questions. @typesafeai 🤝 @convex work great together. @hmartenjoyer @CompleteSkeptic @justKDeng @mikeysee

Ira Bodnar

@irabukht

Jev dropped the price of SEO/GEO fixes by 90% Agents that audit and fix a client's SEO/GEO used to cost us ~$250 Here's where the savings come from: 1/ 30x faster reads of Search Console and PostHog/Mixpanel data 2/ 30x faster checks of what ChatGPT searches on Bing 3/ 30x faster modeling of what users ask Gemini and Claude 4/ 30x faster scans of who ChatGPT and Claude cite 5/ 30x faster analysis of the sources behind those citations 6/ 30x faster gap analysis: why they get cited and we don't 7/ 30x faster fixes across 1,000s of pages on large client sites 8/ 30x faster sorting of which page types ChatGPT cites 9/ 20x faster creation of the pages that make ChatGPT pick you Available in the Ryze AI app and MCP/Claude Connector, link in the 1st comment 👇

jaffa

@dsqjaffa

today i'm releasing Jev for content marketing. still doomscrolling to figure out what to post on social media? that's over now... Jev watches EVERY video in your niche and judges it before it ever reaches you: 1. research: pulls every video in your niche from a database of 12.8M viral videos 2. analyze: Jev watches, studies, and judges each one, the hooks, the formats, the angles, and why they worked 3. create: turns it into a data-backed script, based on proven winners (via Claude) no more guessing on TikTok & Instagram currently available for free in @virlomain + MCP. link below ↓

Florian Darroman

@floriandarroman

Jev is INSANE. I asked 100 Indie Hackers to build a post scheduler with: Jev vs Fable 5.1. The results are unexpected 🤯 (You can clearly see Jev is faster at doing stuff)

Jason Lu

@jasonlu_ai

JEV is changes the world of E2E testing! Same eBay test flow, completed-run medians: Jev: 47s / $0.0067 GPT-5.6 Luna: 62s / $0.0277 Claude Sonnet 5: 79s / $0.4062 Try jev-e2e. github.com/perixtar/jev-e…

Kun Chen

@kunchenguid

almost every day i hear people ask "when should i /compact my session" there's no easy answer because it depends on how likely your future action will need detailed context in the existing window but we have Jev now! introducing compact-adviser - an agent plugin you can use in claude and pi today to help determine whether you're likely at a task boundary that's safe to compact https://github.com/kunchenguid/compact-adviser i built a private eval set from 40 real sessions and manually labeled all the safe vs unsafe checkpoints to evaluate this, and hillclimbed the Jev prompt till it performed quite well i also made it so that the classifier will - optimize for precision (not triggering a compaction prematurely) when context window is small - and gradually shift to optimize for recall (not missing an opportunity to compact) when context window fills up, because the cost of not compacting becomes higher, and at the end the agent will be forced to compact anyway it supports a "hint" mode (just give you a hint and it's up to you to run /compact) vs "auto" mode which runs compaction whenever Jev says it's safe to do so if you have Jev and want to put your compaction on autopilot, try this out and let me know how it goes! support for more harness is coming soon as well

Anand Prasad

@theanandprasad

you can turn any low-cost open-source LLM into a Jev like fast decision model without training it. Here's how to do it: An LLM doesn't write text directly. At every step it scores every word in its vocabulary for how likely that word is to come next, and then it picks one. Those scores are called logits. Suppose you want to know whether a customer email is angry. The usual way is to ask the model, wait while it writes a paragraph, and then try to pull a yes or no out of the response. The trick is to never let it write anything. Give it the email content and the question, and end your prompt with "Answer:". Your full prompt will look something like this: [ Message: "Third time I've contacted you. Still broken." Is this customer angry? Answer yes or no only. Answer: ] The model reads the prompt once and scores every possible next word. You look at two of those scores. "yes" scores 8.1 "no" scores 5.9 Convert those two scores into probabilities and you get 90% yes and 10% no. (Use softmax function -- google it) Your code then uses that number directly. If it's above 0.8, escalate the ticket. Congrats, you have created your own Jev! Here’s why it works: 1. It's fast. Reading a prompt happens in one parallel pass. Writing is the slow part because the model writes one word at a time. Here the model never writes. 2. It's cheap. Output tokens are the expensive ones, and this produces none. 3. There's nothing to parse. You always get a fixed output, and never a paragraph. 4. You get a confidence level along with the answer. A 0.55 means the case should go to a human or a bigger model. A 0.98 means you can act on it. I tried running the code on llama 3.3 70b (via openrouter) and the accuracy was 21/22 (95%). Cost per query: $0.000013, latency: ~500ms

大伟｜AI × Web3

@david1989_zhu

这条新闻最值得警惕的，不是“24小时覆盖Vercel约13%的付费团队”，而是把覆盖率直接当成产品价值。按报道，Jev不写文本，只做判断；这类产品切中的，是开发流程里高频、重复的决策节点。 我没参与Jev落地，下面是基于团队交付经验的判断。当时看到这个数字，我原来会先问：模型到底有多强？后来逐渐发现，团队持续付费更看重一件事：它能否稳定减少人工审核和返工，而不是演示时答得多漂亮。 现在我会先定清决策边界、误判后的回退方式和单次调用成本，再看它是否进入关键链路。“覆盖”来得快是好信号，但留得住、算得过账，才是商业闭环。

amVT

@cruzex100

This is pretty good.. My smoke test earlier: Accuracy: Jev 0.727 Soft accuracy: Jev 0.580 · Laya 0.471 Calibration (ECE, lower is better): Jev 0.144 · Laya 0.213 Latency: Jev ~710ms · Laya ~30-40ms Cost per decision: Jev ~$0.0004 (API) · Laya ~$0 (self-hosted)

Ramarao Kadiyala

@broadrangeAI

Reviewing Jev: fast, structured decisions for AI workflows. Vendor figures: 70–500 ms, $0.042/M input tokens, $0 output. My takeaway: pair low-cost decisions with clear controls and measurable outcomes. Full post: linkedin.com/posts/activity… #AIEngineering #BroadrangeAI

Zhilin Wang

@zhilinjerrywag

#Jev made one thing very clear: Not every AI model needs to generate. Some models should just make really good decisions. So we’re cooking Mev at @metix_ai , a recruiting-native decision model built around job-candidate matching, ranking, and hiring signals. Jev → Mev. More soon.

nocoo

@zhengli

Jev 学费还是要付的，刚开始觉得，每次请求这么便宜，每个月 $5 妥妥够用，一个真实任务下来就教做人：好好研究官方 skill，学会如何正确使用 state 和按分组问问题。

Min(building Moss, hiring now)

@zeroxmin

Great article from Hrishi! So despite all the hype, JEV is a great observer for agent runs (progress, time/cost-to-finish, trace labeling), but not a security gate, and not yet a laziness detector. Still sounds good to plan JEV into our roadmap and explore some use cases for onchain agents!

yuri

@yurinakanishi33

昨日のJevハッカソンで発表した「Jevを使った俳句生成」について訂正があります。 発表では「JevのChoiceで、ひらがなを1文字ずつ生成している」と説明しましたが、間違いでした。 実際には、季語・名詞・助詞・動詞・副詞・結びの定型句などの単語をコード側であらかじめ用意し、それらを組み合わせて5音・7音の完成行候補まで先に作っていました。 Choiceに渡していたのは、その時点の文字列から、いずれかの完成行に到達できる「次の1文字」だけです。 さらに、候補が1つならコードで確定。複数ならChoiceが返した確率の上位3候補を再正規化し、コード側でもう一度抽選していました。そのため、Choiceが選んだ文字を必ず使っていたわけでもありません。 つまり、単語や文章の道筋をコード側で先に用意したうえで、Choiceが返した確率を使ってコード側で分岐する実装でした。申し訳ありません。 添付動画は、コード側で用意していた語彙と単語のつながりに関する制約を外した版です。残した制約は、5・7・5の区切りと、使用できる文字を1字1音として扱える73字に限定する文字種制限だけです。 毎回、ここまでの句、季節・状況・気分などの文脈、残りの音数、73字すべてをChoiceに渡しています。Choiceが返した全73候補の確率をコード側で再正規化し、その確率を重みに次の1字を抽選。これを17回繰り返しています。 結果は......全くできていませんねw #aimeetup

Nitor

@yuntian9999

这几天 X 上一直刷到 Jev。 最反常的一点是：它根本不写字。 TypeSafe AI 把 Jev 定义成第一款公开的 “System One Model”： 你给它一段状态 + 几个明确问题，它不生成长文，而是直接返回选择、评分、概率和置信度。 官方价格： $0.042 / 100万输入 tokens，输出免费。 TypeSafe 还声称，在它针对 System One 任务的工作流里，Jev 可以做到 20–200× 更快、40–400× 更便宜。 注意，这是厂商自己的特定任务测试，不是“全面吊打 GPT”。 所以我觉得它不是 GPT / Claude 的替代品。 更像是给 Agent 加一个“判断层”： GPT / Claude 写和规划 → Jev 判断 / 路由 → 代码执行 真正值得看的不是又一个新模型。 而是 AI 开始分工了。

Vincent.seo

@xunaoo

Jev 가 우리 로직을 대체할 수 있을까요? 아닙니다. 대체가 아니라 로직 안으로 들어앉습니다. 제 브라우저 에이전트에 붙여 봤습니다. 판단 한 단계가 2초에서 0.2초, 모델 호출이 열 번에서 두 번. 그런데 정확도는 6%p 낮습니다.

abdullah a | مبرمج

@xooox888

للمبرمجين 👨‍💻✨ Jev من TypeSafe AI يقدم فكرة مختلفة عن نماذج الذكاء الاصطناعي المعتادة. بدل التركيز على المحادثات وتوليد النصوص، تم تصميمه لمساعدة التطبيقات على اتخاذ قرارات واضحة وسريعة داخل النظام ⚙️🤖 مناسب للمهام المتكررة مثل التصنيف، التوجيه، والأتمتة.

WTR

@wtry1102

話題のJev。 いまいちよくわかってなかったので Astra君に解説ショート動画をつくってもらった なるほど・・・なるほど🙄

Frit🅾️ Pendej🅾️

@vwapster

I think Jev and Laya were the missing pieces to algorithmic trading. Sure you can hardcode gates and adjust variables, but being able to add a real time decision maker within your algorith is a game changer. I can smell and taste an early retirement.

VertrAI

@vertr_ai

The small-LLM fallback for typing is an important detail: Jev chooses actions, while text generation stays with an LLM. We featured your 1x demo, with credit, in our short explainer of that division of labor: x.com/vertr_ai/statu…

VertrAI

@vertr_ai

154ms vs 860ms is a useful routing result, scoped to these 200 synthetic classification cases—not a blanket ChatGPT comparison. We credited your latency and cost charts in our Jev video, alongside the game/browser demos: x.com/vertr_ai/statu…

Vadim Choi

@vadimchoi

$10k/month just to decide which agent should answer. not to answer. to decide. group chat + @mentions: fast, cheap, terrible UX. an LLM orchestrator: nice UX, 4-7s and $0.00046 a message. @typesafeai jev: 0.29s median, $0.00002. same decision, $20/month.

Ayush Jain

@urayushjain

Most AI models are built to generate. But Jev is trying something different: Don’t generate. Decide. Give it some context and a specific question, and it returns a structured decision with a probability. I found this interesting for things like model routing, agent workflows, tool safety and RAG. The idea is simple --Instead of calling a big LLM for every small decision, let your code handle the workflow and use a smaller model where human-like judgment is actually needed. Honestly, this feels like an interesting direction for AI systems.

つぼね👨‍💻

@tubone24

Jevに飽きちゃったのと、全然利用料が安いので、公開してみました。 雨の日はスピードで遊ぼう！！ jev-speed.tubone24.workers.dev

tshmieldev

@tshmieldev

Jev vs LLMS for clasifying tweets in your feed! Try it out: github.com/tshmieldev/sha… Make sure to drop a star!

Baris Terzioglu

@terzioglubrs

Agent memory is usually just an append-only Markdown file that grows forever, and most frameworks load the whole thing into context on every single run. That has been bothering me for a while, but never quite enough to fix it for our agents. Jev feels like it might be a low effort patch to this, by simply scoring each memory against the prompt first, then load only the relevant bits for that specific request.

Petru - Tech Driven

@techdrivenpetru

Yes but not with Jev. I used another classifier, locally, just as fast, just as good. Example: setup a "server" in python that loads the classifier model. Add a hook in Claude that fires on "pre-tool-use" and next time you ask Claude a random question like "how do I lint check a project?" and it tries to freelance and read your entire repo only to intoxicate itself and pollute its context, the classifier will slap its hand, say "no sir, you answer from knowledge" and deny the tool call. I tested this yesterday with success, but need to refine it as it misfires. Basically I was able to identify general queries, instances where I would ask something and Claude would rush ahead and run pip install without me asking or just write code instead of answering. A classifier is hypercheap compared to a regular LLM and would catch all of these. The model I used was DeBERT large. It's still stupid fast, I tested it on an Apple with M1 (regular) and you don't feel it running.

Prasenjit Sarkar

@stretchcloud

I keep seeing the same fix across agent stacks this week: teams are ripping the expensive model out of the middle of their decision loops. TypeSafe shipped Jev in early access on September 15. Not a chat model. They call it a System One model: hand it a state and typed options, it returns calibrated probabilities, no text generation at all. The mechanism is simple. Most agent loops burn a full LLM call on decisions that never needed generation: route to worker A or B, is this result relevant, approve or block this action. TypeSafe claims up to 200x faster inference and 400x lower cost on that class of decision. Ricker's own tests below land at 193x and 444x. What happened next is the real story. Within four days, Cognition's Jared Palmer shipped Kev, a LoRA adapter on Qwen2.5-0.5B trained in 1 hour 45 minutes on a MacBook Pro, with an API close enough to point TypeSafe's own SDK at it. Kev now scales up to a 9B version that trails Jev by about 4.5 points on held out evaluation. Laya-MLX arrived the same week targeting millisecond decisions on Apple Silicon. A community leaderboard, JevBench, already ranks a dozen of these models. The adoption signal convinces me this is not a toy. TanStack AI shipped a native decide() API for typed choices, scores and booleans. Beacon, an open source memory layer, uses Jev to score which coding sessions are worth turning into reusable lessons. Three teams, one primitive, inside a week. RouteLLM out of Berkeley showed back in 2024 that routing simple queries to a cheap model cuts cost over 85% while holding 95% of GPT4 quality, and production semantic routers report 40 to 90% savings today. What changed is that the router stopped being a side project and became a shipped, benchmarked model category with a name. The bottleneck this solves is real: every agent framework has a model sitting in a loop answering questions that never needed a sentence back. My read is that the decision layer becomes as standard a piece of the agent stack as the vector database became for retrieval, and whoever owns the default there owns a lot of the unit economics conversation for the next year of agent infrastructure. https://x.com/0xRicker/status/2101705843200721203

Stanislav Sorokin

@stas_sorokin_

1,000 AI papers sorted into 24 topics for $0.0585. Then Opus 5 graded the labels. @nutlope's Jev paper map went viral, but the pipeline never shipped and the eval was "still running". So I rebuilt both and opened them. The first judge run came back empty: Opus spent its whole budget thinking and answered nothing. Reasoning off, second run: it agreed with Jev on 85 of 100 papers, at 153x the cost and 1.9s against 57ms per paper. The 15 misses are not random. One number Jev already returns tells you which labels to recheck. Cheap models sort. Expensive models audit only what the cheap one flags. Repost if you classify anything at scale, because the eval rows are public and anyone can rerun them with their own judge in one command. Code in the reply.

内田勉 DirecTune.app β公開中

@sidodtv

なんか24時間後に Tiboリセットくるらしいので、今のうちに #directune の開発を進めるよ〜！ まずは、ゆっくり風動画解説で背景が自動作成されるようになったよ。試しに Jevの解説動画を作ったよ DrecTuneは、AIがアシストしてくれる動画生成・制作サービスだぞ‼️

Dibya.shree

@shree_code

Jev doesn't write. It only decides. Yes/no, choices, scores — each with calibrated confidence in 70–500ms. Input is $0.042 per million tokens. Output is free. The first System One model. This is the plumbing agents were missing. firecrawl.dev/blog/what-is-j…

さっと

@satto_sann

Jevでサクッと分類器は価値があるのかビルドトラップできそう

Andrew

@s4yonnara

Jev CEO, Diogo Almeida (ex-OpenAI): "It's not Claude Code because Claude Code is still part of the assistance era." He co-authored GPT-4, ChatGPT, InstructGPT and RLHF. Now he's building what comes next: 200x faster. 400x cheaper. A decision brain for AI agents. His argument is simple: frontier LLMs are insanely good at creating, but we're wasting them on thousands of tiny decisions: Which agent goes next? Is this relevant? Does this need approval? Those calls don't need generation. They need a model built to decide. In 18 minutes, Almeida explains why today's AI was built for assistance, not real automation, and why "tomorrow's AI will be for automation." This makes most $500 agent engineering courses look obsolete. Watch it today, then steal my 10-step guide to giving your agents a decision brain below ↓

s1rozha1

@s1rozha_

Jev hype is not about chat. it’s about decisions everyone is trying to compare it to ChatGPT / Claude / Cursor wrong frame Jev doesn’t need to write you a beautiful paragraph it needs to answer questions like: should this agent act now? which tool should run next? is this signal worth trading? is this market mispriced? should this wallet be ignored or watched? that is the interesting part prediction markets are basically endless small decisions under uncertainty not one giant “AI intelligence” moment thousands of tiny calls: price changed liquidity moved wallet entered news dropped odds lagged edge appeared edge disappeared you don’t want a slow model writing essays for that you want something cheap, fast, typed, and measurable that’s why Jev is interesting. not because it replaces frontier models because it makes the boring decision layer cheap enough to run everywhere

AIセラ｜Obsidian✖️クライアントワークを極めるエンジニア

@ryo_hukugyo_pro

Jevをいかに使いこなすかが今後トークン節約や自動化をするAI社員を大量量産する上で、 めちゃくちゃ重要。

Richard Seroter

@rseroter

I'm reading up on Jev (typesafe.ai/blog/introduci…) and learning how it differs from "regular" classification ML models. I gave a bunch of articles to @googlecloud Gemini Enterprise and asked it for a breakdown. Helpful. Still digging in before going hands-on.

Ranjan Kumar

@ranjankumar

𝐖𝐡𝐞𝐫𝐞 𝐉𝐞𝐯 𝐁𝐞𝐥𝐨𝐧𝐠𝐬 𝐢𝐧 𝐚𝐧 𝐀𝐠𝐞𝐧𝐭 𝐇𝐚𝐫𝐧𝐞𝐬𝐬 (𝐍𝐨𝐭 𝐚 𝐌𝐨𝐝𝐞𝐥 𝐒𝐰𝐚𝐩) Your agent harness needs a decision model. Where you place it depends on one brutal fact: Jev's ordering is trustworthy. Its confidence numbers are not. Most placement guides skip this distinction entirely. They tell you to pick a threshold and execute above it. That works if you actually have a probability. You might not. 𝐓𝐡𝐞 𝐜𝐨𝐫𝐞 𝐩𝐫𝐨𝐛𝐥𝐞𝐦: A model can rank beautifully and lie about magnitude simultaneously. When you gate execution on if confidence >= 0.85: approve_transfer, you are trusting a number that was never calibrated to your prevalence, your cost ratio, or your queue. Move the threshold up and you trade recall you never measured for precision you cannot state. You have no idea how far along an unmarked axis you moved. 𝐓𝐡𝐫𝐞𝐞 𝐝𝐞𝐜𝐢𝐬𝐢𝐨𝐧𝐬, 𝐭𝐡𝐫𝐞𝐞 𝐝𝐢𝐟𝐟𝐞𝐫𝐞𝐧𝐭 𝐧𝐞𝐞𝐝𝐬: Routing and ranking consume ordering only - which answer is best matters, the number beside it does not. Gated execution consumes magnitude - the threshold is your boundary and it must land on a calibrated scale. Relative logic like if top1 - top2 < 0.1: escalate consumes differences - and rescaling the score axis will flatten margins unevenly, breaking your code. The sepsis alert systems of 2020 learned this hard way. Michigan switched off alerts when COVID shifted patient prevalence beneath a fixed threshold. No weights changed. The denominator moved, the promise broke silently, and nurses drowned in false alarms. 𝐓𝐡𝐞 𝐟𝐢𝐱 𝐢𝐬 𝐬𝐞𝐪𝐮𝐞𝐧𝐜𝐞, 𝐧𝐨𝐭 𝐭𝐮𝐧𝐢𝐧𝐠: First question: is this number admissible as a probability at all? Run a calibration study on a few hundred labelled cases from your own queue. Samuel Sacco's measurements from 18 September and Adil Muhammad Pervez's 8,000 judgments both reach the same conclusion: fit your own map. The weights are shared across every account by design - there is no per-customer adaptation - so a deployer-side calibration map is your only mechanism. 𝐒𝐞𝐜𝐨𝐧𝐝 𝐪𝐮𝐞𝐬𝐭𝐢𝐨𝐧: given that the number is calibrated, where does your cost ratio put the line? That part of the familiar advice still works. Run them backwards and you are tuning a dial with no markings. Read the full analysis on calibration, harness placement, and where Jev actually fits: https://ranjankumar.in/jev-system-one-model-agent-harness-placement Follow for more practitioner insights on agentic AI systems and production AI engineering. #AgentiveAI #AIEngineering #SystemOne #Calibration #MLOps #DecisionModels #HarnessEngineering #Jev

Doge 🐶

@pprownets2023

💡 ยกระดับ AI Agent ให้เก่งขึ้น 100x ด้วย Jev Engineering (Decision-Making Brain) เคยเจอปัญหาไหม? เวลาทำ AI Agent แล้วสั่งงานด้วยข้อความธรรมดา (Free-form Prompting) มักได้ผลลัพธ์ที่ไม่แน่นอน ควบคุมยาก และนำไปต่อยอดในระบบแบบอัตโนมัติได้ลำบาก 🤯 เอกสาร JEV Engineering ได้เสนอแนวคิดในการแก้ปัญหานี้อย่างน่าสนใจครับ: 🔹 1. Jev คืออะไร? Jev คือ "สมองส่วนตัดสินใจ" (Decision-Making Model) สำหรับระบบ Agentic Systems ที่ไม่ได้เน้นการคุยแบบ Free-form Chat แต่เน้นการแปลงข้อมูลสถานะระบบ (Structured State) ให้กลายเป็น Typed Outputs ที่ชัดเจน เช่น ช้อยส์การตัดสินใจ, คะแนนความเกี่ยวข้อง หรือคำสั่ง ใช่/ไม่ใช่ พร้อมระบุค่าความน่าจะเป็น (Probability) 🎯 🔹 2. ทำไมถึงทำให้ระบบเก่งขึ้น 100x? การแยกส่วน "คิดตัดสินใจ" (Decision) ออกจาก "การลงมือทำ" (Execution) ช่วยลดความคลุมเครือ และทำให้ระบบเสถียรขึ้นอย่างมหาศาล โดย Jev จะส่งผลสูงสุดในจุดที่มีการตัดสินใจถี่ๆ (High-frequency Decisions) เช่น: 🔍 Research Evaluation: ประเมินความน่าเชื่อถือของแหล่งข้อมูลอัตโนมัติ 🛠️ Action Selection: เลือกใช้ Tool หรือส่งต่อ Agent ให้ถูกตัว ✍️ Content & Draft Refinement: วางแผนและเกลาเนื้อหาเชิงลึก 🚦 Human-in-the-loop Gating: ตัดสินใจว่าเคสไหนต้องส่งให้มนุษย์ตรวจต่อ 🔹 3. สรุปสถาปัตยกรรม 3 ขั้นตอน (How it Works) Input State (01): ใส่ Goal, Context, Rules และ Actions ที่ทำได้ Jev Brain (02): ประเมิน เหตุผล วางแผน และคืนค่าเป็น Typed Decision Orchestrator (03): นำผลลัพธ์ไป Validate และ Execute ร่วมกับ Agent, Tool หรือ Human สดๆ 💡 สรุปสั้นๆ: ถ้าอยากให้ AI Agent ทำงานอัตโนมัติได้อย่างแม่นยำ เลิกส่ง Free-form Prompt สั่งงานตรงๆ แล้วเปลี่ยนมาใช้ Structured State + Typed Decisions เพื่อการขยายระบบที่นิ่งและน่าเชื่อถือครับ!

FHILY👑

@Oluwaphilemon1

Laya is making a pretty strong case for local AI agents. On a 16GB MacBook Air, local Laya is reportedly making decisions in around 45ms. Cloud-based Jev is closer to 300ms per decision. That puts Laya roughly 7 to 11x faster in this Tetris setup. And the hardware is just a MacBook Air. No dedicated GPU server. No API round trips. No per-token bill. Laya runs locally, so once the model is on the machine, the inference cost is effectively $0 per decision. Jev has the opposite tradeoff. You get a cloud model, but every decision depends on the network and the API response coming back. Tetris makes this painfully obvious. The agent has to constantly: see the board → decide → act → see the new state → decide again. When every move takes hundreds of milliseconds, the delay compounds. At 45ms, Laya can keep that loop moving much faster. And there’s another piece I find important: Laya’s weights are open-source. That means you’re not just renting access to an AI agent through an API. You can actually run the model yourself, inspect the setup, and build around it locally. For benchmarks, you can argue about accuracy and scores all day. But put two agents inside a real-time environment and latency becomes impossible to ignore. Same Tetris board. One model waits on the cloud. The other is sitting on your laptop making decisions almost immediately. In this particular test, the difference is pretty brutal. Local inference isn’t just about saving API costs. Sometimes the biggest advantage is simply that the model is already there when you need it.

Vaishnavi

@_vmlops

Someone built a way to run Jev on Cloud Run with an RTX PRO 6000. ~47s cold start. ~117ms end-to-end. 100+ req/s at concurrency 32. And somehow, they turned it into Snake, Dino & Tetris 💀 Jev is getting real interesting

abdel

@AbdelStark

Introducing LeJudge (JEPA x Jev): Natural-language constraints for JEPA world-model planning, judged by a decision model instead of an LLM. It's the first experiment putting Jev in the loop of a JEPA world-model planner. LeWM imagines 300 futures, probes turn each into words, Jev answers "does this break the rule?", code adds the penalty. Rules in plain English. No LLM, no generated text. Results + code + paper 🧵👇

Kiyoro

@0xKiyoro

OPUS 5.5 IS MAKING YOUR AGENT DUMBER RIGHT NOW AND IT WILL NEVER TELL YOU. JEV CAUGHT IT. The default effort on Opus 5.5 dropped from high to medium. If your agent never set effort explicitly, it is now reasoning at a lower setting than the one you tested it on. No error, no warning, every health check still green. Jev sets effort on every single message, so the requests that went through it never changed. The ones that relied on the default did. That gap is how it showed up. The rest of the migration is loud. Each of these returns a 400 on the first call: > thinking: disabled, now rejected. Drop the field and set effort > tool_choice any and tool, removed. Use auto + strict > computer_20251124, retired. Use computer_toolset_20260801 > editing above a thinking block, rejected. Append only, or drop_block You will find all four in five minutes. The effort change you will not find at all. The cache has its own quiet trap. Hop Opus 5.5 to Sonnet 5 and back and a session that cost 3.32 costs 4.36, +31%, because Sonnet cannot read Opus's reasoning. Change effort at the top of a request or switch fast to standard mid-session and the cache is gone again. Jev picks speed once on turn one and sets effort per message, so it survives. The upgrade is still worth it. $4 / $20 per 1M instead of $5 / $25, cache reads at $0.20 instead of $0.50, and 66.4% on Terminal-Bench 4.0 against 57.9% for GPT-6 Astra. Write effort into every request before you touch the model ID. The default stopped meaning what you think.

Arthur Marques

@ArthurLabMRP

Benchmarks suck. So I made Jev and Laya play chess instead ♟️ Insomnia project. Repo in the thread

MalluCuler

@Farhan60291312

Benchmarked Laya locally on an ancient laptop: 💻 i5-5200U (2 cores) | 12GB RAM | 5400 RPM HDD Kept it resident in RAM and got sub-second (~630ms) typed decisions on pure CPU! 🔥 Great open-weight release by @Nandakishorm1. No GPU or cloud LLM needed for fast routing. #laya #jev

Ram Vinjamuri

@RamV2003

(1/5) trying out jev this week with all the hype. kept it apples to apples vs the fast models it competes with: llama 3.1 8b and deepseek v4 flash same labelled inputs. jev: 235ms median, 82% accurate. llama: 497ms, 68%. deepseek flash was accurate but ~2s genuinely cool. then some odd emergent behaviour: no position or label invariance

► Alex Rivas | Joseador y Desarrollador ☻

@alex_rivas_v

🥊Laya vs Jev He diseñado un experimento para probar que modelo es mejor, si Jev o Laya, y lo hice poniendolos a jugar ajedrez. La verdad es que ambos son bastante tontos para esto, y no va de mejorar prompts, aunque seguiré probando algunas formas de mejorar las partidas.

netrunner

@plotarmordev

Another Jev competitor just dropped: CLM-8B from @jackyk02 and the Stanford/NVIDIA team, with open weights under Apache 2.0. They claim it's comparable to Jev with up to 9x faster inference We went from zero options to a new one almost every day!

Edwin Mesa

@edwinfmesa

Seguí con dos consultas: un código de verificación que no llega y cómo navegar con el teclado. Jev eligió seguridad y manual de usuario; Laya, privacidad y términos. En ambos casos me convencen más las elecciones de Jev, aunque Laya volvió a responder más rápido con mi setup.

Edwin Mesa

@edwinfmesa

Empecé a escribir: "¿Qué precio tiene la suscripción anual?". Antes de pulsar Enviar Jev y Laya ya sugerían planes_y_precios.md. Eso es lo que quería probar: si el agente puede tener una pista de qué consultar antes de que le llegue la pregunta, con mi setup Laya fue más velóz

Packapun

@packapun

New kid on the block typesafe.ai/blog/introduci…

Nikkhil Narang

@acenik10

Your agent doesn't need a bigger brain. It needs a faster one. Most agent decisions are tiny — urgent or not, which queue, yes/no. We run them through frontier models anyway. TypeSafe's new Jev does it in milliseconds for a fraction of the cost. #AIAgents #Jev

Tuan N

@__tuan____

Sooner or later everyone will come to realize System One Paradox or Jev Paradox is real. The inevitable path forward is System One Plus or System One+. x.com/__tuan____/sta…

Han Keo

@KeoGrowth

En regardant Jev et Recursive Mass, je peut être sûr d'une chose les LLM ne sont pas la voie royal pour être le cerveau des agents, du moins pas pour longtemps. Jev même n'est pas encore au top du top sur ces probabilité montre déjà ces preuves pour ses prises de décision (plus rapide que des LLMs) Et Le Recursive Mass (pensées latentes sous forme de nombres) montrent qu'il est plus efficace pour les agents de communiquer entre eux partageant leu états internes plutôt que générer du texte compréhensible par des humains. Une chose est clair, on ne doit pas utiliser les IAs pour nos agent commet on les utilises pour nos discussions. Les Modèles n'ont pas fini d'évoluer :)

Jim Merioles

@jimmerioles

Yep, best ELI5 on TypeSafe's Jev:

Zenko | NeuralFrame Labs

@NeuralFrameLabs

Local Laya beats Jev 100% #localai #ainews #ai #laya #jev

𝗿𝗮𝗺𝗮𝗸𝗿𝘂𝘀𝗵𝗻𝗮— 𝗲/𝗮𝗰𝗰

@techwith_ram

TypeSafe Founder's Blueprint for Building with Jev. Check out the pdf here: drive.google.com/file/d/17h982x…

TypeSafe AI

@typesafeai

LLMs distilling Jev like

ASI Hub

@ASIHubHQ

DEEP DIVE REPORT: TYPESAFE AI LAUNCHES JEV SUB 100MS SYSTEM 1 DECISION ENGINE EXECUTIVE BRIEFING Routing routine classification tasks through a 70 billion parameter language model is architectural malpractice. TypeSafe AI has introduced Jev, a sub 100 millisecond deterministic decision engine proving that single pass probabilistic heads dramatically outperform bloated models on System 1 tasks. 1. THE ARCHITECTURAL DILEMMA Enterprise AI architectures suffer from catastrophic inference latency. Standard multi turn agentic loops invoke heavy reasoning models for simple deterministic gates, resulting in multi second roundtrips and unsustainable cloud compute costs. Jev restructures decision topology by executing fixed logit classification in single forward passes at 70 millisecond latency. 2. FOUR HUNDRED TIMES COST EFFICIENCY Empirical benchmarks demonstrate 400x reduction in compute expenditure per decision token. By constraining entropy to predefined domain spaces, Jev eliminates model hallucinations while providing mathematically verifiable confidence scoring across decision boundaries. 3. 2027 TO 2030 OUTLOOK By 2027, over 80% of autonomous software agent workflows will execute on dedicated System 1 kernels like Jev. Frontier multi modal models will operate purely as supervisory reasoning nodes, invoked only when high entropy creative synthesis is strictly required. Strategic takeaway: Compute efficiency dictates architectural survival. Split your system into fast reflex execution and slow deliberative reasoning.

GeethanTech

@GeethanTech

Jev returns bounded choices, scores and probabilities instead of prose. That limits out-of-schema output, not incorrect decisions. TypeSafe’s speed and cost claims still need independent testing. geethantech.com/posts/jev-type…

CRYPTOFANZ💜💎

@cryptofanz12

𝗝𝗲𝘃 𝗜𝘀 𝗧𝘂𝗿𝗻𝗶𝗻𝗴 𝗔𝗜 𝗜𝗻𝘁𝗼 𝗔 𝗦𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲𝗱 𝗗𝗲𝗰𝗶𝘀𝗶𝗼𝗻 𝗟𝗮𝘆𝗲𝗿 ⚡ As the first System One model launched by @typesafeai, Jev is designed for software that needs AI to make fast, structured decisions—not generate paragraphs of text. Traditional workflow: Prompt → Text → Parsing → Validation → Application Logic Jev takes a more direct route: Application State → Typed Question → Typed Decision No JSON prompting. No complex output parsing. Just typed decisions that applications can use directly, along with probability and confidence. Jev supports three decision patterns: 🔹 Choice — Select from predefined options 🔹 Score — Evaluate using a defined scoring framework 🔹 Noul — Handle structured decision problems built around application logic Speed is another key feature, with response times designed around roughly 70–500ms. Its pricing is also aimed at high-frequency inference: 💰 $0.042 per million input tokens 🆓 Output tokens free Potential use cases include: 🎫 Ticket routing 🛡️ Content moderation 📊 Risk scoring 🤖 Agent branching and workflow decisions The bigger idea is simple: AI doesn't always need to write something for a human to read. Sometimes, software just needs a fast, reliable decision. That makes models like Jev an interesting direction for AI agents, automation, and real-time application infrastructure. Available through the http://B.AI API as: Jev-1.13.0 Jev-Latest 🔗 http://chat.b.ai/chat @justinsuntron @BAI_AGI #TRONEcoStar #AI #Agents #B_AI

ABNewswire

@ABNewswire

Jev Brings Fast, Structured Decision-Making to AI Workflows dlvr.it/TVcnJg #ComputersSoftware #Technology

Lilys.ai

@LilysAI_

@tetumemo 要約ノートができました👉 lilys.ai/digest/1147040… - Jevは次の行動を決めるためのAIエージェント用「脳」。 - 文章生成ではなく、選択・数値・はい/いいえの判断に特化。 - うまく設計すれば、品質を落とさずコスト削減が狙える。

echild

@eboppu

some more tests. jev for tool filtration is really good.

James Pardoe

@JamesPardoe

If you're still scratching your head about Jev, this is the best explanation that I've seen.

Dorian Smiley

@dsmiley411

Testing Jev’s accuracy tonight for next best action prediction. The results: Canonical accuracy: 98.6% Generalization accuracy: 41.5% We ask Jev to predict the next state in a program from the current partial program. The suite has 25 cases and we ran it 20 times. Seven cases are represented in the in context examples. The other 18 are held out cases that require Jev to generalize from those examples. The failures are not uniformly random. Jev generalizes perfectly on some unseen compositions and fails almost deterministically on others. That suggests there may be specific structural boundaries to what it can infer from context. Maybe some of this is prompt design. Maybe it is a capability boundary. We’re testing that now. But next best action prediction is important. A huge amount of software today contains really brittle decision logic: onboarding, payments, claims, revenue cycle, approvals, exception handling, etc. If Jev is a bet on software consuming intelligence, this is exactly the kind of high frequency, high value logic it needs to improve.

Jason Alco

@Jasonalco

I put Jev vs 5.6 Sol head-to-head on a text categorization step in a platform I’m building. Surprised by the outcome. Aren’t text-based judgements the exact use case Jev is made for?

Mendy

@MilkMendy

If you run a business, Jev might be a really good alternative to your heavier models. It's really good at classifying things and choosing between yes or no, and it could be 200X quicker and a lot cheaper than the other frontier models. #AI #AIAgents #AIAutomation

Sodeh Abadi

@irSodeh

Jev vs Laya-MLX isn't really a rivalry. Jev is the original, commercial model. Laya-MLX is a local, open-source take on the same idea, running natively on your Mac. Same core concept: "System One" models. No chatting, no text generation. Just fast, typed decisions. #Jev #Laya

WquGuru

@wquguru

Jev 发布才 8 天，开源替身已经一堆了，今天诺基亚也来蹭热闹了😅 我在 MacBook 上，让官方 Jev 和 3 个开源方案同跑 20 张中文工单🫱 Laya 25ms，比 Jev 快 50 倍 djev 509ms 官方 Jev 1.3 秒 AnyJev 将近 4 秒 仓库都在这： Laya（Mac 版）http://github.com/mizorewww/laya-mlx djev（Mac 版）http://github.com/jamescorbett/mlx-vlm AnyJev http://github.com/nokia-applied-research/AnyJev 但快的，不一定对 👇

Darshan Jain

@i_darshanjain

Jev is all over my timeline rn 👀 Went looking into it and stumbled across Laya. Open-source System 1 decision model. They even have a Jev comparison 👀 7.8× faster. 3× better calibrated. 45/51 languages. Self-hosted. And the weights are right there. 😂 Yeah… I am watching this space. https://huggingface.co/convaiinnovations/laya

Saksham Sharma

@code_saksham

And just when I thought I have a crazy idea with JEV that i will open source then this happened. Please @typesafeai fix it ASAP, the curiosity inside me to build that thing is insanely high

Chase

@0xChaseTM

Jev Engineers just released Jev, and the architecture is simple: state goes in, auditable decisions come out I mapped the decision layer into a field blueprint: what Jev reads, what it returns, and how an agent uses those answers in a real loop: step 1 → give each layer a job. The LLM plans and writes. Jev chooses between bounded options. Code carries out the action. step 2 → build the state before asking for a decision. Include the goal, evidence, available actions, previous actions, and what the environment looks like now. step 3 → define the questions in code. Choice selects an option. Score rates it. Noul estimates whether a condition is true. The answers are typed, so the runtime can use them directly. step 4 → ask independent questions together. One state snapshot can support routing, risk, and relevance decisions in the same pass. step 5 → use the answers to route work. A lookup, a complex reasoning task, and a tool call don’t need the same model or execution path. step 6 → add a gate before the tool runs: allow, ask, or deny. A high-confidence answer still has to pass the action policy. step 7 → execute the selected action and check the outcome. Record what the tool actually did, including failures. step 8 → update the state with that result. Otherwise, the agent’s next decision is based on a world that no longer exists. step 9 → apply the pattern where small decisions repeat: model routing, tool guardrails, inbox triage, research classification, browser actions, and context compaction. step 10 → judge the whole task. Track whether the agent reached its goal alongside cost, latency, approvals, and failed actions. the result: you can trace an action from the evidence Jev received to the decision it returned and the outcome your tools produced. Save the Jev Decision Infrastructure blueprint for your next agent build ↓

Utkarsh Maheshwari

@utkarshgsuv

Is the Jev hype real, so I made to test this on my movie recommendation engine, which is getting crazy results on Quen Re ranker , I used an open-source alternative of Jev , Laya. But unfortunately Quen is way better, dekhlo! @typesafeai #jev #Jev

Xiang Wei

@xgwei

Great minds! I've been building a stylized SF with Opus too, just without the pets. My cars follow rules on the real street network, lanes and traffic lights included. Jev driving pedestrians is tempting though. How many agents can it run at once?

spect

@spectnfa

The founder of Jev just dropped a 1-hour masterclass on how Jev actually works LLMs → Decisions → Verification → Coding Agents 0% → 0:40 - why LLMs alone aren't enough 25% → 20:27 - the 3 primitives behind Jev 50% → 26:15 - turning huge AI tasks into tiny reliable decisions 75% → 45:04 - real systems: data, agents, verification 100% → 56:33 - the future of coding agents, memory and subagents Most people are still learning prompts. This explains the layer that decides what the model should do, checks whether it worked, and decides what happens next. Basically: how you go from “AI chatbot” to software you can actually trust to run itself. And it's explained by the guy building it. Bookmark this. Watch the full hour tonight. Then read the guide below and steal the architecture for your own agents ↓

silentguy

@silentguyy66

Grok Bot does the job, Jev decides where the job goes next That's the ai company split: a work model and a decision model In this free 22-min tutorial they walk through jev, typesafe's "system one" model that makes decisions instead of chatting You ask one question, Jev answers from a fixed menu: noul, choice, score The demo: customer messages in, frustration score out. usable right in code Not another chat model, it's the fork in the pipeline Grok Bot keeps generating Jev keeps deciding Humans keep the approvals The teams wiring in that third layer are quietly becoming one-person companies Save this before your competitors put a decision layer between their bots

Geek Lite

@QingQ77

把本地 MLX 推理的 Laya 模型和托管 API 的 Jev 放在同一个 T-Rex 跑酷赛道上对打，用延迟、存活和回放数据对比本地推理与云 API。 github.com/virajbhartiya/…

Just AI

@thelazyyogii

上次说的 Jev： 不会陪聊，只负责拍板。 这几天又冒出个 Laya。 活差不多，路子不一样。 一起想成： 流水线上专门判题的人， 不写长文，只答「过不过 / 归哪类」。 Jev = 外面请来的裁判 开箱就能用，按量付钱， 选项多、材料长时更稳。 适合懒得自己搭、先要结果的人。 Laya = 自己厨房里养的裁判 权重能下，Apache 开源，本机跑， 更快更省隐私。 但要自己微调；选项一堆时容易糊。 适合有机器、要数据不出门、愿自己调的人。 一句人话： 不是谁取代谁—— 要省事找 Jev，要自己掌控找 Laya。 都是选择题机器，不是聊天搭子

Arthur Marques

@ArthurLabMRP

was trying to figure out how to make these 2 models fight each other, so I put jev vs laya in chess lol script picks the 10 best moves, captures, and moves that put pieces under attack. still wip but jev feels way smarter

Morgan

@morganlinton

It has been a really interesting experience to build an eval suite for System One Models like Jev. My first two attempts didn't work out. But I learned a lot in the process, and since Jev is so fast, and cheap, it's a lot easier to run quick tests than with an LLM. I'm on attempt number three with this eval suite, and thought I'd share more about it. The eval suite is called @VulcanBench Verdict, and here's the high-level on how it works: