Show HN: BitRun – Run 1-bit LLMs on CPU at 98T/s without compiling

Category: ai-ml

Tags: llm-inference, ternary-models, cpu-optimization, terminal-ui, model-runner

Score: 6.8/10 (Innovation: 6, Technical: 7, Documentation: 8, Utility: 6)

BitRun is a zero-compile runner and terminal dashboard for 1-bit and 1.58-bit ternary LLMs on commodity CPUs. It simplifies model management, provides a live telemetry cockpit, and serves an OpenAI-compatible API, making low-power CPU inference accessible without complex toolchains.

Target audience: AI researchers, edge computing developers, and backend engineers interested in efficient local LLM inference

Repository: https://github.com/IshekKhal/bitrun · Go · MIT · 8 stars

View on Hacker News