Show HN: NerfWatch() – Track AI degradation with daily tests and community votes

Category: ai-ml

Tags: ai-benchmarking, model-monitoring, llm-evaluation, typescript, cli-tool

Score: 6.0/10 (Innovation: 5, Technical: 6, Documentation: 8, Utility: 5)

NerfWatch is a self-hosted testing harness that benchmarks AI models for performance degradation ('nerfing') by running a fixed set of 100 auto-graded questions across reasoning, logic, code, instructions, and long-context tasks. It supports multiple providers, provides statistical baseline comparisons, and pairs test scores with community voting on the hosted site.

Target audience: AI engineers, ML researchers, and developers monitoring model quality over time

Repository: https://nerfwatch.lol · TypeScript · MIT

View on Hacker News