Show HN: Livenerf – a benchmark for whether Opus 5.5 gets nerfed

Category: ai-ml

Tags: benchmark, llm-evaluation, model-drift, claude, statistical-testing

Score: 6.8/10 (Innovation: 7, Technical: 6, Documentation: 8, Utility: 6)

Livenerf is a pre-registered, longitudinal benchmark designed to detect whether Claude Opus 5.5 quietly degrades after its launch. It uses a frozen panel of 78 sometimes-correct questions, runs daily through Claude Code, and applies statistical drift detection inspired by Anthropic's own error-bars paper. The project is notable for its rigorous methodology and transparency in tracking model performance over time.

Target audience: AI researchers, ML engineers, and technically-minded users monitoring frontier model consistency

Repository: https://github.com/ninjahawk/livenerf · Python · MIT · 1552 stars

View on Hacker News