Two decision models make different calls on the same security queries.
Edwin Mesa
@edwinfmesa
Seguí con dos consultas: un código de verificación que no llega y cómo navegar con el teclado. Jev eligió seguridad y manual de usuario; Laya, privacidad y términos. En ambos casos me convencen más las elecciones de Jev, aunque Laya volvió a responder más rápido con mi setup.
Jev + Kimi K3 for fraud detection!
TLDR: Jev classified 100 emails in 1.42 seconds, then I routed the uncertain cases to Kimi K3. The full pipeline got 96/100 correct for only ~$0.07.
Video is not sped up, check out the live run!
Here was my process:
I gave Jev 100 emails to classify (a mix of 50 legit & 50 fraudelent emails). It classified all of them in 1.42 seconds.
An underrated feature about Jev is it will give you the confidence score for a classification, so I routed any prediction under 95% confidence to Kimi K3 to be fully sure.
31 emails fell below that threshold. After routing those to Kimi K3, the combined pipeline reached 96% accuracy.
The full run took 16 seconds & ~$0.07 in inference costs:
- $0.068 from Kimi K3 on @togethercompute
- $0.003 (1/3 of a cent) from Jev on @typesafeai.
I think this is a really interesting pattern: use a fast specialized model like Jev for the narrow task, then route the uncertain cases to a larger LLM.
I feel like this kind of approach could be a game changer for use cases like fraud or anything realtime. You can use the speed & low cost of Jev while having a larger LLM as a fallback to ensure high accuracy.
Tested TypeSafe’s Jev (no-text, probability-only model) as an AI agent safety monitor.
Checking each action first worked well caught most attacks with almost no false blocks, and much faster than Gemini.
I don’t know if people still remember Jev. Things move so fast here.
I’ve spent the last few days testing it, and found a few things I think builders should see.
Prompt injection is one of the most interesting things to test in AI, so I made a little demo to show how it could change Jev’s answer.
Jev is a great model. I just want people to know what to watch out for when they use it.
