GitHub · by kyle-pena-nlp
BeelineBench
How well do decision models plan across a multi-step problem?
Kyle Pena built BeelineBench to measure whether a decision model can plan ahead. The model gets a problem such as Blocksworld or a sliding tile puzzle and a list of actions, and it picks the next action until it reaches a solution. The score is the shortest path divided by the number of states the model explored, so a perfect score means the model went straight to the solution. The benchmark has 7 domains with 100 trials each, and a write-up on LessWrong.
kyle-pena-nlp/beelinebench on GitHub
- Stars
- 3
- Forks
- 0
- Language
- Python
- Last push
- Oct 12, 2026
Repository created Oct 9, 2026. Counts read from the GitHub API on Oct 12, 2026, and they move daily — quote them with the date.
