Show HN: Shrewd – what I learned distilling LLM labels into local classifiers

Category: ai-ml

Tags: llm-distillation, text-classification, calibration, active-learning, local-inference

Score: 6.8/10 (Innovation: 6, Technical: 7, Documentation: 8, Utility: 6)

Shrewd distills repeated LLM classification calls into small, fast local models (tf-idf, embeddings, SetFit, fine-tuned encoders) with calibration, confidence-based routing back to the LLM, and a locked test set for honest eval. It also shares measured lessons on prompt optimization, label acquisition, and threshold setting. The combination of teacher-student distillation, decision-panel-style multi-question outputs, and statistical routing bounds is a practical, well-engineered package.

Target audience: ML engineers, data scientists, and backend developers who repeatedly run LLM classification and want cheaper, faster, local inference

Repository: https://github.com/sshah03/shrewd · Python · MIT · 3 stars

View on Hacker News