Show HN: Jevstiller – Distill Jev into a local model, with a disagreement bound

Category: ai-ml

Tags: model-distillation, proxy, local-inference, api-cost-reduction, statistical-guarantees

Score: 7.0/10 (Innovation: 7, Technical: 7, Documentation: 8, Utility: 6)

Jevstiller is a drop-in proxy that distills a hosted classification API (Jev) into a local model on the fly, with a statistical guarantee that local answers agree with the API within a user-set disagreement budget. It monitors agreement via a random audit slice and falls back to the API when unsure, achieving 60-70% local answer rates at 99%+ agreement with ~20x latency reduction.

Target audience: backend developers and ML engineers using hosted classification APIs

Repository: https://jevstiller.pages.dev/posts/the-guarantee/ · Python · Apache-2.0 · 70 stars

View on Hacker News