Show HN: GLM-5.3 744B at 4 tok/s on a MacBook Pro, experts streamed from 4 SSDs
Category: ai-ml
Tags: moe, ssd-streaming, llm-inference, macos, benchmarking
Score: 7.0/10 (Innovation: 7, Technical: 8, Documentation: 8, Utility: 5)
ArgoDrive provides layout, balancing, and instrumentation tools for running large mixture-of-experts models (up to 744B parameters) from SSDs on a single MacBook Pro. It achieves up to 2.69x faster prompt processing by selectively reading only routed experts and splitting reads across multiple drives, addressing the real bottleneck of per-layer read latency rather than aggregate bandwidth. The project is notable for its rigorous benchmarking methodology and honest, detailed documentation of reproducible gains.
Target audience: ML engineers, researchers, and hobbyists running large language models on consumer hardware
Repository: https://github.com/argonautlabsai/argodrive · Python · MIT · 1 stars
View on Hacker News