Show HN: Relay – a self-hosted LLM gateway with smart routing and request pacing

Category: ai-ml

Tags: llm-gateway, routing, api-gateway, self-hosted, openai-compatible

Score: 6.0/10 (Innovation: 5, Technical: 6, Documentation: 7, Utility: 6)

Relay is a self-hosted LLM gateway that provides a unified OpenAI-compatible API across many model providers, with smart routing, request pacing, fallbacks, and usage tracking. It aims to simplify managing multiple LLM backends, especially for agent-based applications, and includes a built-in dashboard and optional local classification model.

Target audience: backend developers, AI engineers, DevOps

Repository: https://github.com/anchorshell/relay · Go · NOASSERTION · 7 stars

View on Hacker News