Show HN: Imp, an LLM Server for RTX 50 GPUs, Written by Claude Code
Category: ai-ml
Tags: llm-inference, cuda, gpu, openai-api, quantization, mixture-of-experts, blackwell
Score: 7.0/10 (Innovation: 7, Technical: 8, Documentation: 8, Utility: 5)
Imp is a C++23 LLM inference server engineered specifically for NVIDIA Blackwell sm_120a GPUs (RTX 50 and RTX PRO Blackwell), exploiting native FP4 tensor-core math and PCIe expert offload to serve MoE models larger than VRAM. It exposes OpenAI and Anthropic-compatible APIs and reports competitive benchmarks against vLLM and llama.cpp on a single RTX 5090. The narrow hardware targeting and one-maintainer/no-CI-GPU situation limit its reach, but the technical depth and documentation are notable.
Target audience: AI engineers and developers running local LLMs on single Blackwell consumer/prosumer GPUs
Repository: https://github.com/kekzl/imp · C++ · MIT · 47 stars
View on Hacker News