Show HN: Enki LLM A pure rust (even kernels) with live GPU/CPU brain migration

Category: ai-ml

Tags: rust, llm-inference, vulkan, gpu-compute, spir-v, heterogeneous-computing

Score: 7.0/10 (Innovation: 8, Technical: 9, Documentation: 6, Utility: 5)

Enki LLM is a pure Rust implementation of Llama-3.2 1B inference that compiles Rust kernels directly to SPIR-V via the Enki runtime, eliminating CUDA dependencies and enabling live migration of model weights and KV-cache between GPU VRAM and CPU RAM mid-generation. The project demonstrates a novel heterogeneous compute approach with cross-silicon portability and zero-copy pointer-like memory access on the GPU.

Target audience: systems programmers, ML infrastructure engineers, GPU compute researchers

Repository: https://github.com/enkiruntime/enki_llm · Jupyter Notebook · 8 stars

View on Hacker News