I LEAKED THE JEV STACK I RUN MY AGENTS ON, IT CAUGHT 11 OF THEM IN ONE NIGHT BEFORE THEY BROKE SOMETHING
Agents don't fail at writing code, they fail at deciding, which file to open, which hunk matters, whether to run that command, whether they're actually done.
Every one of those comes back as a paragraph you have to parse, and none of them come back with a number telling you how sure it was.
So I made them typed, a confidence on every answer, and a gate written in code, not in the model, that acts, holds, escalates or refuses.
Ran it on my own repo last night, 30 tasks, 4 agents in parallel, 360 judgments at 311 ms each, four cents for the whole thing.
19 shipped with nobody watching, 11 got stopped cold.
Those 11 are the whole product.
We built a plugin that gives Jev a browser in Cline, and have been blown away by the results.
1. Install it in our new desktop app: Customize > Marketplace > Plugins > search 'jev-browser'
2. Create a Vercel AI Gateway API key, then save it to ~/.cline/plugins/cline-jev-browser.config.json as {"gateway": {"apiKey": "..."}} and restart Cline.
3. Ask any browser task and it will launch Chrome in the background to complete it.
We just ran Jev on our WebMCP benchmark.
The result: basically broke the benchmark.
Jev + Mercury 2.5 (a fast, low-cost LLM) using WebMCP solved 100% of the tasks at roughly 112× lower model cost than GPT-6 Astra using computer use with code execution. Compared to Astra using screenshot-based computer use, the model cost was 245× lower (!).
We also compared Jev operating the browser with and without WebMCP.
We used Browser Use’s open-source Ultrafast, with some improvements to the harness to make it more reliable across the benchmark.
Jev’s browser-control accuracy on its own was not amazing - adding WebMCP nearly doubled the number of solved tasks, from 25/49 to 49/49, while reducing model cost by 18% (more on why below).
The benchmark and methodology are fully open and reproducible.
Full results: https://webmcp.com/benchmark
A few words on how the Jev + WebMCP harness works and why this is exciting:
Jev receives text as input and a set of discrete options it can choose from. With WebMCP, those options are the tools exposed by the website. At each step, Jev sees the task, the available tools and previous results, then picks what to do next.
The limitation is that Jev can’t generate arbitrary text, which you need for tool arguments. For example, it can choose the search_products tool, but it can’t generate the search query itself.
So we split the work: Jev picks the tool and Mercury 2.5 generates the arguments if needed.
This works well because turns out most of the cognitive load in these tasks is around choosing the right action. The argument generation itself is relatively simple, so we can delegate to a small and very fast model. We used Mercury, which outputs 1,000+ tokens/sec and is very cheap.
The result is a pretty simple combination: Jev for tool selection + Mercury for arguments + WebMCP for the interface. It ends up being very reliable, very fast, and very cheap.
A few words about Ultrafast and why do we think it underperforms:
Without WebMCP, Jev chooses from the page’s controls: which button to click, which field to fill, or which option to select.
But choosing a valid button is different from choosing the right next step. The agent still has to navigate menus, understand forms, recover from errors and recognize when the task is actually complete.
Our hypothesis is that WebMCP makes the decision space much simpler. Instead of figuring out a sequence of clicks through a website, Jev chooses explicit actions that directly advance the task.
@typesafeai itself documents weaker accuracy on questions requiring multiple reasoning steps. WebMCP moves much of that complexity into the website’s tools, leaving Jev with clearer decisions and fewer opportunities to go wrong (in a sense WebMCP "compresses" a sequence of clicks into one tool call).
Our modified Ultrafast setup solved 25/49 tasks - that is a result for our particular implementation and benchmark, not a universal limit on Jev or Browser Use. We are open to more harness optimization to get this result to perform better, feel free to directly contribute to the benchmark here: https://github.com/nekuda-ai/WindTunnel
Browser-use ultrafast: https://github.com/browser-use/jev-ultrafast
we built blazing fast computer/browser use with Jev + @Stagehanddev.
this task cost $0.001 and executed at near instant speed (in a remote browser btw)
the loop: observe the page, send a11y tree as state + actions as questions, Jev decides the next action, then Stagehand executes it.
