Show HN: BeelineBench – How well do decision models solve multi-step problems?

Category: ai-ml

Tags: benchmark, decision-models, planning, search, ai-evaluation

Score: 7.5/10 (Innovation: 7, Technical: 8, Documentation: 9, Utility: 6)

BeelineBench is a benchmark for evaluating how efficiently decision models solve multi-step planning problems by scoring how directly they navigate to a solution. It uses a best-first search harness across seven domains (e.g., sliding puzzles, Blocksworld, Wikipedia navigation) and compares model performance against classic heuristics, random choice, and an oracle. The project is well-documented, configurable, and integrates with custom model APIs, making it a valuable tool for AI planning research.

Target audience: AI researchers, ML engineers, and developers evaluating planning capabilities of decision models

Repository: https://github.com/kyle-pena-nlp/beelinebench · Python · Apache-2.0 · 3 stars

View on Hacker News