Show HN: Popular Claude Code skills vs. a placebo: 2 beat it, 1 did worse

Category: devtools

Tags: placebo-controlled, ai-agents, benchmarking, claude-code, statistical-analysis

Score: 6.5/10 (Innovation: 7, Technical: 6, Documentation: 8, Utility: 5)

This project conducts placebo-controlled trials of popular Claude Code skills, comparing them against neutral text of similar length to determine if they genuinely improve coding agent performance. It provides a rigorous, pre-registered methodology and detailed statistical analysis, revealing that most skills offer no measurable benefit over a placebo. The work is interesting for its experimental approach to evaluating AI tooling and its potential to inform developers about effective practices.

Target audience: AI researchers, backend developers, DevOps

Repository: https://github.com/simonether/skill-placebo · Shell · MIT · 17 stars

View on Hacker News