Show HN: Modeljury – a jury of cheap LLMs that escalates the hard cases

Category: ai-ml

Tags: llm, ensemble, human-in-the-loop, moderation, python

Score: 5.8/10 (Innovation: 5, Technical: 5, Documentation: 8, Utility: 5)

Modeljury is a Python library that queries multiple cheap LLMs and uses their agreement as a confidence signal, escalating disagreements to human review with dissenting reasons. It includes an honest benchmark against a premium model, showing cost/accuracy tradeoffs with transparent losses. The practical value is turning ensemble disagreement into an actionable review flag for classification and moderation tasks.

Target audience: backend devs and ML engineers building LLM-powered classification or moderation pipelines

Repository: https://github.com/YasserManss/modeljury · Python · MIT · 1 stars

View on Hacker News