Show HN: Strot – Compile an LLM's intent once, run decisions locally
Category: ai-ml
Tags: llm, local-inference, decision-making, compiler, cpu
Score: 7.5/10 (Innovation: 8, Technical: 8, Documentation: 8, Utility: 6)
STROT compiles a natural-language decision task into a frozen Python artifact using an LLM once, then uses a small local model (144M params, CPU) for fast, deterministic, typed decisions without runtime API calls. It innovates by moving LLM work off the hot path, adding calibration, abstention, and a golden-record promotion gate to ensure reliability.
Target audience: backend developers, data engineers, ML engineers
Repository: https://github.com/strot-ai/Project-STROT · Python · MIT · 43 stars
View on Hacker News