Show HN: SDCGuard – Inspect vLLM KV-checksum coverage per request
Category: devtools
Tags: llm-inference, vllm, integrity-checks, testing, kv-cache
Score: 6.0/10 (Innovation: 7, Technical: 7, Documentation: 6, Utility: 4)
SDCGuard is a release-testing tool for LLM inference servers that inspects vLLM's KV-cache checksum coverage per request. It replays recorded runs to flag requests whose prompts were only partially checksum-verified, catching silent wrong answers that still return HTTP 200. The project demonstrates real reproduction of a vLLM bug and an insightful analysis of checksum coverage gaps.
Target audience: ML infrastructure engineers, backend devs working on LLM serving
Repository: https://github.com/hari-kancharla/sdcguard · Python · MIT
View on Hacker News