Show HN: BitRun, 1-bit LLM runner on CPU (100T/s on Apple Silicon)

Category: ai-ml

Tags: inference, llm, quantization, cpu, terminal-ui, bitnet, go

Score: 6.3/10 (Innovation: 5, Technical: 6, Documentation: 8, Utility: 6)

BitRun is a standalone Go-based runner and terminal dashboard for 1-bit and 1.58-bit ternary LLMs on commodity CPUs. It wraps a native inference engine with zero external dependencies, offering model management, an OpenAI-compatible REST server, and a live hardware/energy telemetry cockpit. It's interesting for making low-power ternary inference accessible without complex build toolchains.

Target audience: backend developers, ML engineers, edge/embedded developers, researchers

Repository: https://github.com/IshekKhal/bitrun · Go · MIT · 8 stars

View on Hacker News