Show HN: Phobos – a tiny kernel language that now runs LLMs on a 2019 GPU

Category: ai-ml

Tags: gpu-kernels, llm-inference, compiler, cuda, rust

Score: 7.5/10 (Innovation: 8, Technical: 9, Documentation: 7, Utility: 6)

Phobos is a tile-based GPU kernel language, compiler, and LLM inference engine written in Rust. It achieves competitive or better performance than llama.cpp on low-VRAM and mixture-of-experts models by streaming experts from host memory without manual CPU/GPU splits. The project combines an MLIR-based compiler, an OpenAI-compatible inference server, and a distributed clustering runtime, making it a technically ambitious full-stack systems project.

Target audience: systems programmers, compiler engineers, ML infrastructure engineers, and developers running LLMs on consumer GPUs

Repository: https://www.joa-ebert.com/posts/2026-10-06-phobos-inference/ · Rust · Apache-2.0 · 5 stars

View on Hacker News