Show HN: Gauze fixes (some) open-weight LLM deficiencies

Category: infrastructure

Tags: llm, gateway, reliability, openai-api, local-inference

Score: 6.3/10 (Innovation: 6, Technical: 6, Documentation: 8, Utility: 5)

llm-gauze is an HTTP gateway that sits in front of a local OpenAI-compatible LLM to detect and remediate common failure modes like retries, hung models, empty responses, context overflows, and malformed tool calls. It logs every exchange and applies configurable fixes before the client sees the result, making it useful for anyone running open-weight models locally.

Target audience: backend devs, devops, AI engineers running local LLMs

Repository: https://github.com/ggeorgovassilis/llm-gauze · Python · MIT · 2 stars

View on Hacker News