Show HN: Janus – Go binary that runs GGUF models via Vulkan on AMD/Intel/Nvidia

Category: ai-ml

Tags: llm, local-inference, openai-api, go, vulkan, gguf

Score: 6.0/10 (Innovation: 5, Technical: 6, Documentation: 7, Utility: 6)

Janus is a single Go binary that runs GGUF LLM models locally via Vulkan (AMD/Intel/NVIDIA) or CPU, exposing an OpenAI-compatible API for use with tools like Cursor and Cline. It simplifies local inference by eliminating Python, Docker, and Ollama dependencies, with features like hot-swapping models and thinking-model support. While not introducing novel algorithms, it fills a practical niche for developers wanting a lightweight, cross-vendor local LLM server.

Target audience: backend developers, AI engineers, and developers using local LLMs with OpenAI-compatible tooling

Repository: https://github.com/Vibra-Ingenn/Janus · Go · MIT · 135 stars

View on Hacker News