X · by morganlinton
An eval suite for System One models
Building the tests Jev-like models need.
Morgan
@morganlinton
It has been a really interesting experience to build an eval suite for System One Models like Jev. My first two attempts didn't work out. But I learned a lot in the process, and since Jev is so fast, and cheap, it's a lot easier to run quick tests than with an LLM. I'm on attempt number three with this eval suite, and thought I'd share more about it. The eval suite is called @VulcanBench Verdict, and here's the high-level on how it works:
Sep 23, 2026 · 31 likesOpen on X
morganlinton is building an eval suite for System One models like Jev.
Open the source
