Show HN: A 3KB LLM inference engine in C
Category: ai-ml
Tags: llm-inference, size-optimization, c, arm64, safetensors
Score: 7.0/10 (Innovation: 8, Technical: 9, Documentation: 8, Utility: 3)
bytesized is an extremely compact LLM inference engine written in C, compiling to a 3,156-byte binary that runs SmolLM2-135M with exact token-for-token matching against Hugging Face transformers. It reads original safetensors and tokenizer files without conversion, using mmap and a clever BPE implementation that avoids reading 48,900 merge rules. The project demonstrates impressive systems programming, including self-unpacking, custom math functions, and a 420-byte assembly unpacker.
Target audience: systems programmers, embedded developers, AI hobbyists
Repository: https://github.com/utsav1033/bytesized · C · MIT
View on Hacker News