Show HN: JevBench, a reproducible benchmark for typed decision models

Category: ai-ml

Tags: decision-making, calibration, certification, llm-inference, benchmarking

Score: 8.5/10 (Innovation: 8, Technical: 9, Documentation: 9, Utility: 8)

Decisio is an open-source server that answers typed questions about text with calibrated probabilities from a single forward pass, avoiding text generation. It allows task-specific calibration and certification using small labeled datasets without retraining, and it achieves competitive performance on JevBench while running on local hardware.

Target audience: AI/ML engineers, data scientists, backend developers

Repository: https://benchmarkheaven.com/jev-models · Python · Apache-2.0 · 7 stars

View on Hacker News