GitHub · by Ishida Lab
ProbabilityBench
Tests Jev and other decision models against exact probabilities.
Ishida Lab’s benchmark builds simulated worlds where the true probability of each outcome is known, so a model’s reported probability can be checked against it directly. There are three worlds: peer review, covert transfer and credit risk. Each case has a readable memo, a sampled outcome and the exact probability. The fixed test set is on Hugging Face.
ishida-lab/ProbabilityBench on GitHub
- Stars
- 4
- Forks
- 0
- Language
- Python
- Last push
- Oct 15, 2026
Repository created Oct 14, 2026. Counts read from the GitHub API on Oct 15, 2026, and they move daily — quote them with the date.
