Show HN: Nerftracker – We show you when models get nerfed
Category: ai-ml
Tags: ai-benchmark, model-evaluation, sandbox, reproducibility, coding-agents
Score: 5.0/10 (Innovation: 5, Technical: 6, Documentation: 4, Utility: 4)
NerfTracker benchmarks AI coding models by running them in isolated, clean macOS sandboxes to detect performance degradation (nerfing) over time. It records sessions, scores outputs with hidden tests, and provides side-by-side video demos, offering a rigorous and reproducible evaluation method.
Target audience: AI researchers, ML engineers, and developers evaluating LLM coding capabilities
Repository: https://nerftracker.com
View on Hacker News