Show HN: Carmen – testing AI-written GPU kernels
Category: ai-ml
Tags: gpu-kernels, apple-silicon, metal, llm-codegen, benchmarking, testing
Score: 6.5/10 (Innovation: 7, Technical: 7, Documentation: 8, Utility: 4)
Carmen is a harness that has an LLM write Metal GPU kernels for Apple Silicon, then rigorously tests each kernel with hidden inputs, compares it against MLX, and measures whether it actually speeds up a real model. Its most interesting contribution is methodological honesty: it plants bugs to validate its own tests (catching 75/75 vs 40/75 for naive checks) and reports that impressive single-kernel wins mostly vanish in end-to-end model inference due to Amdahl's law.
Target audience: machine-learning engineers and Apple Silicon performance researchers
Repository: https://github.com/utsav1033/carmen · Python · MIT · 1 stars
View on Hacker News