Show HN: A StarCraft BW Arena where LLMs play by writing code

Category: ai-ml

Tags: starcraft, llm-benchmark, ai-agents, game-ai, bwapi

Score: 6.8/10 (Innovation: 7, Technical: 8, Documentation: 6, Utility: 6)

StarSkirmish Bench evaluates LLMs by having them write StarCraft: Brood War bots in C++ within a one-hour time limit, then compete against human-written bots. It leverages BWAPI, a mature open-source API for StarCraft AI, and uses OpenBW for headless simulation. The benchmark tests long-horizon reasoning and agentic coding, with scores scaled against known bot strengths.

Target audience: AI researchers, LLM evaluators, game AI developers

Repository: https://starskirmish.com/bench/ · C++ · LGPL-3.0 · 1713 stars

View on Hacker News