Show HN: jevals – replacing LLM judges with typed Jev decisions
Category: ai-ml
Tags: evals, llm, agents, decision-models, guardrails
Score: 7.5/10 (Innovation: 8, Technical: 7, Documentation: 8, Utility: 7)
jevals is a Python library that replaces LLM judges with typed decision models (Jev-style) for evaluating agent traces. By packing multiple yes/no, choice, and rubric questions into a single forward pass, it drastically reduces cost and latency, enabling evals on every trace and inside the agent loop. It supports various backends including hosted APIs and local models, and offers built-in evals for tool use, grounding, relevancy, and security.
Target audience: AI/ML engineers, backend developers building LLM agents
Repository: https://github.com/openlayer-ai/jevals · Python · MIT · 105 stars
View on Hacker News