Show HN: Reflex – a cold-start GGUF engine that runs Aleph Alpha's Kolibri-1
Category: ai-ml
Tags: gguf, inference-engine, cuda, cold-start, rust, serverless
Score: 7.3/10 (Innovation: 7, Technical: 8, Documentation: 8, Utility: 6)
Reflex is a Rust and CUDA inference engine for GGUF models that optimizes for cold start time rather than throughput, using ahead-of-time compiled kernels to achieve ~448 ms cold starts. It targets serverless GPU workloads where per-second billing makes rapid startup critical, supporting multiple architectures including Aleph Alpha's Kolibri-1.
Target audience: AI/ML engineers, backend developers, serverless platform users, devops
Repository: https://github.com/lateos-ai/reflex · Rust · MIT · 63 stars
View on Hacker News