Show HN: Benchmark coding agents on deploying apps

Category: devtools

Tags: benchmark, ai-agents, cloud-deployment, coding-assistants, evaluation

Score: 7.0/10 (Innovation: 7, Technical: 7, Documentation: 8, Utility: 6)

ai-deploy-bench is a benchmark that measures how well AI coding agents (specifically Claude Code) can build and deploy five types of web apps across five cloud providers. It runs each cell in GitHub Actions, verifies deployments with black-box tests, and publishes results including success rates, human intervention needed, security scores, and costs. This provides a reproducible, end-to-end evaluation of agent capability from code generation to live deployment.

Target audience: AI researchers, platform engineers, DevOps teams, and cloud providers evaluating agentic deployment workflows

Repository: https://github.com/cloudishai/ai-deploy-bench · Python · 3 stars

View on Hacker News