Show HN: Parsimony – an open-source benchmark for how much code AI models add

Category: ai-ml

Tags: benchmark, ai, code-analysis, swe-bench, parsimony

Score: 6.8/10 (Innovation: 7, Technical: 6, Documentation: 8, Utility: 6)

Parsimony is an open-source benchmark that quantifies how much code AI models add when solving software engineering tasks. It parses patches from public benchmarks like SWE-bench, counts coding units (syntax elements), and ranks models by net code added relative to peers, emphasizing code parsimony. The project provides a reproducible, transparent methodology and a leaderboard website.

Target audience: AI researchers, ML engineers, software engineering researchers, and developers evaluating AI coding assistants

Repository: https://parsimony.lasu.dev/ · Python · MIT · 12 stars

View on Hacker News