Show HN: I Made 17 AIs Play 7,700 Hands of Poker
Category: ai-ml
Tags: benchmark, poker, llm-evaluation, decision-making, game-theory
Score: 6.3/10 (Innovation: 6, Technical: 6, Documentation: 7, Utility: 7)
This project benchmarks 17 AI models (LLMs, decision models, and a simple script) playing thousands of hands of heads-up poker, using duplicate deals and all-in adjustment to reduce luck. It reveals surprising findings: a 30-line script beats decision models and draws with frontier models, while cheap reasoning LLMs match expensive ones at a fraction of the cost. The benchmark provides practical insights for choosing AI for sequential decision-making under uncertainty.
Target audience: AI researchers, ML engineers, and developers evaluating LLMs for decision-making tasks
Repository: https://chrystianschutz.com/blog/ai-poker-holdembench/
View on Hacker News