Jev model routers and tool routing

Stanislav Sorokin

@stas_sorokin_

Jev raised $870M at $7.5B yesterday. On 262 of our real decisions it beat Claude Haiku 5.5, 204 to 177. TypeSafe's decision model got that valuation weeks after launch, in the same week Anthropic shipped Haiku 5.5 at $0.10 per million input tokens. So we checked whether a cheap general model can do the deciding. Haiku 5.5 got the labeled holdout cases from 17 decision systems we run every day, with the same instructions, word for word. → Right picks: Jev 204/262, Haiku 5.5 177/262 → By system: Haiku won 1, tied 5, lost 11 → Cost of the 262 calls: $0.058 on Haiku, $0.022 on Jev (2.7x) → Median answer: 1.16 s vs 375 ms → Biggest gap: "which builder gets this task" 6/12 vs 10/12 The twist: the one system Haiku won is the one that asks for judgment. "Did the agent really finish the task": Haiku 11/11, Jev 9/11. One caveat: those instructions were tuned on Jev for weeks. Haiku read them cold. Would you put a general model or a decision model behind your agent's routing? Repost this now, because "Haiku is cheap enough for everything" is turning into a default nobody measured. Follow @stas_sorokin_ for numbers from AI systems running in real production.