On-device audio, no LLM in the loop

More like this

• 

  Riley Brown

  @rileybrown

  Yeah Jev by @typesafeai is very cool. It classified 500 emails in seconds. And it costed 3.5 cents.

  

  X·Triage and routing

  500 emails for 3.5 cents

  Emails

  500

  Cost

  3.5 cents

• 

  vogel

  @ryanvogel

  this model is actually insane at email classification i tested it on 1500 of my own emails to see how well it works and I am blown away

  

  X·Triage and routing

  Triage across 1,500 emails

  Volume

  ~1,500 emails

• 

  Hassan

  @nutlope

  Jev + Kimi K3 for fraud detection! TLDR: Jev classified 100 emails in 1.42 seconds, then I routed the uncertain cases to Kimi K3. The full pipeline got 96/100 correct for only ~$0.07. Video is not sped up, check out the live run! Here was my process: I gave Jev 100 emails to classify (a mix of 50 legit & 50 fraudelent emails). It classified all of them in 1.42 seconds. An underrated feature about Jev is it will give you the confidence score for a classification, so I routed any prediction under 95% confidence to Kimi K3 to be fully sure. 31 emails fell below that threshold. After routing those to Kimi K3, the combined pipeline reached 96% accuracy. The full run took 16 seconds & ~$0.07 in inference costs: - $0.068 from Kimi K3 on @togethercompute - $0.003 (1/3 of a cent) from Jev on @typesafeai. I think this is a really interesting pattern: use a fast specialized model like Jev for the narrow task, then route the uncertain cases to a larger LLM. I feel like this kind of approach could be a game changer for use cases like fraud or anything realtime. You can use the speed & low cost of Jev while having a larger LLM as a fallback to ensure high accuracy.

  

  X·Triage and routing

  Pick

  Fraud detection with Jev and Kimi K3

  Emails

  100 in 1.42 s

  Correct

  96/100

  Cost

  ~$0.07