Jev question types: Choice, Score and Noul

Which type to use

Pick the type whose answer your code can act on directly. A Choice between refund, rebook and information maps onto three code paths, a Score of frustration onto a threshold, and a Noul onto an if.

Use a Noul for a yes/no judgment and a Score for a position on a spectrum. TypeSafe's docs ask "Is the candidate strong in Python?" of four CVs. As a Noul the answers run 0.03, 0.14, 0.81, 0.92. As a Score with four described levels they land at 0.0, 1.0, 2.05 and 2.89, each on or near a level someone wrote. A Noul of 0.5 means yes and no are equally likely. It does not mean "medium".

Source: Noul, Choose a question type.

Choice

A Choice has a type, instructions (the question) and criteria, which is a map from option name to description. The model gets both the names and the descriptions, so write descriptions that tell the options apart. Use null when a name says it all. Give the full list rather than a shortlist; each extra option costs a few tokens. Add an other or none of the above option when the list might not cover every input.

{
  "state": "I was charged twice for my subscription this month.",
  "model": "jev-latest",
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this?",
      "criteria": {
        "billing": "Payment or subscription issues",
        "technical": "Bugs or integration problems",
        "sales": "Pricing or plan questions"
      }
    }
  }
}

{
  "model": "jev-1.13.0",
  "answers": {
    "department": {
      "type": "choice",
      "choice": "billing",
      "probabilities": { "billing": 0.88, "technical": 0.12, "sales": 0.0 },
      "confidence": 0.81
    }
  }
}

The 0.81 is TypeSafe's figure. The published formula gives 0.82 on these rounded probabilities.

Past 255 options, or for a deep taxonomy, chain Choices level by level and walk the tree in code. TypeSafe's hierarchical classification cookbook keeps the best few paths at each level rather than committing to one. For a flat list that is too long, score it in windows and then choose within the winner.

Sources: Choice, hierarchical classification cookbook.

Score

A Score's criteria is an ordered array of level descriptions. A level's number is its position in the array, starting at 0. The model sees only the descriptions and judges each level against the state separately, so it never sees the numbers. The score that comes back is each level number times its probability, summed. It can land between two levels. legend maps each number back to its description, and the Python SDK keys legend and probabilities by integer rather than by string.

{
  "frustration": {
    "type": "score",
    "score": 1.05,
    "legend": { "0": "Calm", "1": "Frustrated", "2": "Very angry" },
    "probabilities": { "0": 0.0, "1": 0.95, "2": 0.05 },
    "confidence": 0.92
  }
}

Here the score is 0 × 0.0 + 1 × 0.95 + 2 × 0.05 = 1.05. The same score can hide different answers: 1.0 can mean all the weight is on level 1, or half on level 0 and half on level 2. Read probabilities and confidence alongside it. A score is a position, not a measurement. Threshold it, sort by it, or round it to a level, but don't interpolate an exact quantity between two levels; TypeSafe says jev-1.13's levels are weak in numerical calibration. To combine Scores of different lengths, first divide each by its top level number (the number of levels minus one).

Sources: Score, jaggedness: maths using score.

Noul

A Noul has instructions and optional criteria with true and false descriptions of what a yes and a no mean. The answer is one number, the probability of yes. There is no separate confidence because, with only two outcomes, that number already says everything: near 1 is a clear yes, near 0 a clear no, and near 0.5 means the model is unsure.

{
  "is_urgent": { "type": "noul", "noul": 0.95 }
}

The middle two are where a threshold in your code decides. How to set it is on Jev confidence.

Source: Noul.

Instructions and criteria

instructions is the question. criteria is the set of possible answers: options for a Choice, levels for a Score, and an optional definition of yes and no for a Noul. Treat the criteria as an extension of the instruction. When they pull in different directions, for example a Noul whose true describes a no, jev-1.13 does worse.

Both fields take a string, an object or an array. Start with strings. Switch to an object when one string is doing several jobs:

• Two options keep getting confused. Give each one fields for what it covers, what belongs to a neighbouring option instead, and a few examples. Use the same field names on every option. The names are yours; none are reserved, and the model reads them, so keep them short.
• Part of the question comes from your code. Put the database row in its own field and refer to it by name in backticks, rather than splicing it into a sentence. TypeSafe's Noul example compares one CV against several candidate records this way, with one question per record and all of them in a single request.
• The state has several parts. Point the question at one with a backticked path such as support.tickets[0].message.

Examples steer the model, and higher confidence doesn't prove a description is better. On TypeSafe's Safari bug report, adding a matching example to each Score level moved the answer from 1.43 at confidence 0.35 to 1.03 at 0.96. An example unrelated to browsers left it at 1.43 and 0.35. Choose examples whose correct level you already know, then test on different inputs. One builder measured this: winnow's author scored three wordings of the same question against 97 blind hand labels. The structured version (what / not for / examples) and the one-line version were both clean below 0.1. A stricter one-line wording was overconfident: 23% of its lowest bin turned out to be needed.

Sources: Advanced: structure, Score: structured levels, jaggedness: contradictory, winnow README.

Writing Score levels

• Describe situations, not degrees. "Broken or degraded feature, but workaround exists" gives the model something to match against; "moderately severe" doesn't.
• Numbers don't help. Each level is judged on its own, so "worse than the previous level" means nothing, and neither do numbers in the descriptions. In TypeSafe's test, number-only levels split a cosmetic bug between 0 and 1. With descriptive levels it scored 0.0 at confidence 1.0.
• One dimension per Score. "Punctual and smart and experienced" measures three things. Split it into three Scores and weight them in code.
• Give a rare extreme its own level when you would act on it differently, such as "abusive or threatening" above "very angry".
• Use as many levels as you can describe distinctly, up to 10. Three is fine.
• If there is no in-between, use a Choice or several Nouls.

firehose-judge's questions.json shows both styles side by side. Its bait Score describes situations, from "Just talking, no hook" to "Full ragebait or outrage farming". Its sentiment Score uses degree words, from "Very negative" to "Very positive". By TypeSafe's advice, the second is the one to rewrite first if its answers drift.

Sources: Score: writing good levels, firehose-judge questions.json.

How to write a good question

1. One snap judgment per question. TypeSafe's test is whether a knowledgeable person could answer it in a second. "Does this message convey urgency?" passes. "Analyze this message and determine the best course of action" doesn't.
2. Split broad questions. TypeSafe's docs replace "Is message spam?" with six Nouls: does it ask for a password, does it claim an unexpected reward, does it pressure the reader to act quickly, does the sender's name conflict with the email domain, does the link's domain conflict with it, and does the link text hide where it goes. Each answer can then be inspected, weighted and tuned.
3. Write the whole question in instructions. The ID safe_to_publish tells the model nothing, because the model never sees it.
4. For a Noul, one condition and a clear boundary. "Is the customer angry and asking for a refund?" is two Nouls. Phrase it so that a high value means yes: "Does the message contain personal data?", not "Is the message free of personal data?". Words like "any" remove the middle ground. A statement ("The customer is requesting a refund") works as well as a question. Try both, and try with and without criteria.
5. Say exactly what you mean. jev-1.13 reads scoping words, negations and implied conditions at face value. When you catch yourself explaining what you meant after a wrong answer, that explanation is the missing half of the instruction.
6. Send only what the question needs. Unrelated text in the state costs accuracy.
7. Keep questions and thresholds in one file. These are what a reviewer needs to read. TypeSafe's agent-skill page adds: "Agents aren't great at writing questions, so expect to edit collaboratively with them."

Sources: primitives, How to build with TypeSafe, Writing a Noul question, literal reading, agent skill.

A first call, line by line, is on how to use Jev.

Ask the questions you might need

Questions in one request share the state and are evaluated independently and in parallel. One answer is never context for another, and adding a question costs its tokens and barely changes the response time. So ask everything your code might need, including questions that only matter for some inputs, and ignore the answers you don't use. TypeSafe calls this speculative fan-out. In its parallel-questions cookbook, 13 questions in one call were 12.2× cheaper and 10.0× faster than 13 separate calls, with the same answers.

Make a second request only when your code cannot build it until it has the first answer. That happens when the answer decides what to fetch, what the state is made of, or which options the next question offers.

Sources: Speculative fan-out, parallel questions cookbook, When one question depends on another.

Common mistakes

Sources: Jev 1.13 jaggedness, primitives, Score, Noul.

Question sets you can read

Each of these keeps its questions in one file you can open: firehose-judge's questions.json mixes two Choices, two Scores and four Nouls about every Bluesky post, and Bouncer's policy is a YAML file of plain-English questions and thresholds.

GitHub·Security and abuse

firehose-judge

The live Bluesky firehose, judged post by post, with a lane for humans.

LMLeo Mata

0~$0.00003

Akshay 🚀

@akshay_pachaar

GitHub·Benchmarks and evals

jev-as-judge

Traces

10 frozen

Questions

4 in one request

GitHub·Social feeds

LinkedIn NoSlop

A Chrome extension that blurs low-value LinkedIn posts.

Ssushrutb17

0

GitHub·Security and abuse

Jev Anti-Spam Bot

A Telegram bot that deletes only the spam Jev is sure about.

NKNikita Kolmogorov

16

GitHub·Security and abuse

Bouncer

A judgment layer that checks every Claude Code tool call against your policy.

CClownware

1~$0.04/day

GitHub·Context and memory

winnow

A context sieve for Claude Code: Jev judges each tool result before it lands.

GDGhaleb Dweikat

103

Many questions in one request

Questions in a request are answered independently and in parallel, so a long rubric costs little more than one question. Rob Hallam asks 61 about a draft post for $0.0004, and Movez asked 14 yes-or-no questions of each of 100,000 posts for $0.67.

Rob Hallam

@robj3d3

Jev + SuperX = virality solved ✅ Every post gets 61 questions in ~1s for $0.0004 🤯 > fitted on 9,481 real posts from 207 creators > picks the viral post 2 in 3 times > never rewards reply bait So: write, score, rewrite, stop when it peaks. Free, no signup. try it below ↓

X·Social feeds

Post scoring with SuperX

Questions per draft

61

Latency

~1 s

Cost

$0.0004

Movez

@0xMovez

I just built a Jev X Viral Post Analyser. 100,000 viral X posts. 20.4 seconds. $0.67. Claude Opus 5, same corpus, same clock, got through 214 posts and spent $0.98. per post that is ~680x cheaper the full Opus pass would have run $458. viral analysis is the perfect Jev job. • it is not writing, it is 14 yes/no calls per post: > does the hook open a loop, > is there a number in the first line, > is the proof real or claimed. classification, not prose. • what it found: 1,220 posts broke into the top 1%. baseline 1.22%. > superlative claim - 2.34% viral. 1.92x baseline > contrarian take - 1.59%. 1.31x > launch / tool drop - 1.46%. 1.19x and numbered lists, the thing everyone writes: 0.55%. below baseline. the most used hook is the least viral one. full stop. • what you are watching: left is the post under analysis, right is Jev answering 14 typed questions about it, each with a confidence score. the run stops at 20.4s because that is when Jev finished all 100k. pulled the corpus through a few X APIs, one parallel pass into Jev. should I drop it to public? Read my latest article on Jev Engineering below and turn your ideas into reality.

X·Social feeds

Pick

100,000 viral posts in 20.4 seconds

Posts

100,000

Time

20.4 s

Cost

$0.67

Ian Nuttall

@iannuttall

I gave Jev 3,282 of my X posts across 100M views and asked it to find what actually works for growth. 4,252,330 tokens $0.1282 for the full 8m 34s run! Each post got 8 questions about the topic, hook, tone, whether it teaches something, etc. How-to posts got 150 median likes vs the average median of 44. AI and coding was a 1.9x multiplier topic compared and SEO, despite recent posts, was right at base median 1.0x - surprisingly. The recommended topic + angle + voice formula was: AI coding + teach something + provocative

X·Social feeds

3,282 posts, eight questions each

Posts

3,282

Cost

$0.1282

Run time

8 m 34 s

Dan Shipper

@danshipper

we almost never test new foundation models but we've been testing this for ~a week @every and it's pretty wild. the kind of things that will be obviously indispensible in 6-12 months it doesn't produce words as output, it produces probabilities. so it can efficiently act as a judge in cases where you'd need a Fable-level model—but in our testing was 25x faster and 600x lower priced excellent vibe check by @hammer_mt on @every: https://every.to/also-true-for-humans/mini-vibe-check-typesafe-s-jev-judged-everything-i-ve-written-in-0-7-seconds?utm_cta_source=home_main_a_3

Article·Benchmarks and evals

Pick

Every’s editorial vibe check

Judgments

1,709

Total cost

<$0.01

Median per passage

0.35 s

Steve Krouse

@stevekrouse

typesafe's jev is fun! live demo you can play with: typesafe-demo.val.run

Live site·Social feeds

TypeSafe Typewriter

Questions per call

16

Ackerman

@Yarilo7brigada

98% of my feed is junk. Now I don’t even see it I built a filter that reads my feed for me. It runs on Jev - a model that doesn’t generate text; it just makes binary decisions, in milliseconds and for pennies. For every post, it evaluates four questions: Is it relevant to my niche? Does it provide real value? What format is it (breakdown, news, ad, meme)? And does it make loud claims with zero proof? Out of 1,000 posts, only 20 remained. 97 milliseconds per post. 1.2 cents for the entire morning. The most frustrating takeaway, nearly half of my feed was ads and memes, not the creators I originally followed for substance. An hour of mindless scrolling turned into five lines over breakfast.

X·Social feeds

A feed filter that kept 20 of 1,000 posts

Per post

97 ms

Morning cost

$0.012

Kept

20 / 1,000

Tools that write, lint and test questions

Agent skills that know the three types, and tools that check a question set before and after it ships: jevkit lints one offline, jev-calibrate tunes criteria against labels you already have, and Sutro's jev-align optimises the question itself.

GitHub·SDKs and integrations

jevkit

A Rust CLI for Jev, with a linter that checks questions before you pay.

AFAriel Frischer

4

Skill·Benchmarks and evals

tenbin

Split a judgment into Choice, Score and Noul, lint it, then measure it.

SIShingo Imota

4

GitHub·Benchmarks and evals

jev-calibrate

Tune your Jev questions against your own labels, then confirm on held-out data.

Ssmkrv

32

GitHub·Benchmarks and evals

jev-align

Turns human labels into calibrated Jev classifiers with GEPA.

SSutro

305

Skill·SDKs and integrations

Building with Jev

A skill for writing and fixing programs that call Jev.

DBDrew Breunig

153

Skill·Agents and browsers

Augustus

An agent skill for designing systems around Jev’s judgments.

BMBasit Mustafa

11

GitHub·SDKs and integrations

mcp_jev

A local MCP server that runs ready-made Jev question packs.

PKPedro Knigge

1

Where to go next

Once the questions are right, the next decision is what to do with an unsure answer: Jev confidence. The failure modes behind the mistakes above, with the builds that work around them, are on Jev limitations. For whole jobs built from these questions, see Jev for classification and Jev as a judge. To set up a coding agent that writes Jev calls for you, see Jev prompts.