Show HN: Long term Memory and 50M token window for LLM
Category: ai-ml
Tags: llm, kv-cache, memory, inference, deduplication
Score: 6.0/10 (Innovation: 6, Technical: 7, Documentation: 4, Utility: 7)
Galahad-KV is a KV-cache grafting and deduplication library that claims to provide a 50M token context window and long-term memory for LLMs by reusing byte-exact KV cache segments across requests. It integrates with vLLM, SGLang, and llama.cpp and is backed by several arXiv papers, though the README is largely badges and links to papers rather than usage documentation.
Target audience: ai engineers, ml researchers, backend devs working on LLM inference
Repository: https://huggingface.co/papers/2610.10845 · HTML · 48 stars
View on Hacker News