Show HN: A new engine to run Kimi K3 on a laptop
Category: ai-ml
Tags: mixture-of-experts, inference-engine, model-streaming, local-llm, edge-ai
Score: 7.3/10 (Innovation: 8, Technical: 8, Documentation: 7, Utility: 6)
WASTE is a C-based inference engine that runs massive Mixture-of-Experts models like Kimi K3 (2.78T parameters) on consumer laptops by streaming only the required experts from an NVMe-optimized container, keeping the dense parts in RAM. It achieves 0.32 tokens/sec on a 64GB MacBook Pro, proving that full open-weights models can run locally without distillation or specialized hardware. The project includes conversion tools, benchmarks, validation against PyTorch, and an OpenAI-compatible server.
Target audience: AI researchers, ML engineers, and privacy-conscious developers interested in running large language models locally on consumer hardware
View on Hacker News