Skip to content
Made with Jev

X post · by Milind S

Computer use without screenshots

On-device segmentation and OCR turn the screen into text for Jev.

Milind S

@milindlabs

Okay so Jev can actually do computer use really well Without any screenshots, or LLMs and no Pixels leave my mac I dont even read the Dom elements A local CoreML model segments every button and UI element on screen. On-device OCR reads the labels. That text is all Jev gets. It returns a probability across those elements and tells me the best one to click. Then it clicks, re-runs detection, and decides again. In a loop until the goal is done. ~90ms per decision. Faster than any LLM computer use I've tried. Blazing fast computer use, without any latency @typesafeai is building something really interesting

Sep 17, 2026 · 1.4K likesOpen on X

Milind S runs computer use on a Mac without sending screenshots anywhere. A local CoreML model finds every button and UI element on screen, on-device OCR reads the labels, and that text is all Jev gets.

Open the source

More like this