inference on Hacker News

105 titles from 87 authors in the last 30 days, with 94 distinct links and 11 Show HN.

Counts titles, it does not confirm facts. Open the source before concluding anything.

Projects on this term

The score comes from our model.

• Show HN: Resin, fast native scheduler for LLM inference engines · score 7.8

  Resin provides a Rust-based scheduler and KV cache for LLM inference engines, separating scheduling logic from model-specific forward pass kernels in Python. It supports advanced features like speculative decoding, parallelisms, CUDA graphs, and an HTTP serve layer, allowing rapid development of custom inference engines with competitive performance.

• Show HN: Golang DL/ML primitives with inference, diffusion, vision, systemone · score 7.8

  dlgo is a pure Go implementation of LLM inference, multimodal vision, and text-to-image diffusion pipelines, eliminating Python and CUDA dependencies. It supports GGUF models, Vulkan GPU acceleration, and includes a sprite generation tool, demonstrating a significant engineering feat in the Go ecosystem.

• Show HN: Spinf-serve – inference server for bulk scoring and decisions · score 7.0

  spinf-serve is an inference server that scores many questions against a single content read in one forward pass, returning calibrated probabilities per option rather than generated text. It targets bulk classification and decision-making workloads, with prompt packs, batch-invariant scoring, and an OpenAI-compatible API, and benchmarks favorably against vLLM for many-questions-per-document jobs.

• Show HN: Gutsy, subsecond decision model inference on CPU (Jev Style), no API · score 7.0

  Gutsy is a fine-tuned 0.8B parameter model (based on Qwen) that provides calibrated probability outputs for decision-making tasks, running locally on CPU via llama.cpp. It offers a Jev-style API for yes/no, choice, and score questions, with strong calibration and deterministic results, positioning itself as a private, cost-free alternative to API-based decision models.

Recent titles

1. Sam Altman on local inference and use of RAM [video]
   2026-11-01 · youtube.com · HN discussion
2. Fruth – A semantic state inference engine for AI
   2026-11-01 · github.com · HN discussion
3. Inference engine in game scripting language using macros
   2026-10-31 · dasllama.io · HN discussion
4. Sovereign Inference Mesh – Phone-native AI inference and edge compute on Android
   2026-10-31 · zenodo.org · HN discussion
5. Cheaper AI inference is making it harder to profit from renting out compute
   2026-10-31 · tomshardware.com · HN discussion
6. Local Inference with LLMs
   2026-10-31 · jacquesmattheij.com · HN discussion
7. GAAC: Lossless KV Cache Compression for Faster LLM Inference
   2026-10-31 · arxiv.org · HN discussion
8. Estimate and track the environmental footprint of GenAI models at inference
   2026-10-31 · ecologits.ai · HN discussion
9. Show HN: Spinf-serve – inference server for bulk scoring and decisions
   2026-10-30 · github.com · HN discussion
10. OpenRouter rebuilt provider routing for better, faster inference
    2026-10-30 · openrouter.ai · HN discussion
11. Learn LLM inference system on Apple Silicon
    2026-10-30 · github.com · HN discussion
12. Ask HN: What inference engine and KV offload engine do you use and why?
    2026-10-30 · HN discussion
13. What inference engine and KV offload engine do you use and why?
    2026-10-30 · HN discussion
14. Sam Altman: local inference is a terrible allocation of scarce resources
    2026-10-30 · linkedin.com · HN discussion
15. An Open Field Guide to Inference Engineering
    2026-10-29 · docs.modelplane.ai · HN discussion
16. Inference for Developers
    2026-10-28 · particle.ai · HN discussion
17. Cerebras and Oracle Team Up to Bring Cerebras AI Inference to OCI
    2026-10-27 · cerebras.ai · HN discussion
18. Adaptive UI and docs using in-browser inference
    2026-10-27 · passo.uno · HN discussion
19. There are now 5 inference providers for abliterated DeepSeek
    2026-10-27 · allmodels.io · HN discussion
20. Show HN: Resin, fast native scheduler for LLM inference engines
    2026-10-26 · github.com · HN discussion
21. Local Inference on a Raspberry Pi
    2026-10-24 · keepitwiel.github.io · HN discussion
22. Apple Silicon Inference Engine: Kapitsa
    2026-10-24 · github.com · HN discussion
23. Show HN: Inference Cooperative, user-governed AI
    2026-10-24 · inference.coop · HN discussion
24. Private LLM Inference with Homomorphic Encryption [pdf]
    2026-10-23 · vldb.org · HN discussion
25. Cutting our inference bill by 68% by being obsessive about optimization
    2026-10-23 · blog.cloudflare.com · HN discussion
26. Humming: Fast, Flexible Kernels for Quantized Inference
    2026-10-23 · vllm.ai · HN discussion
27. LithosAI Tops Artificial Analysis: Inference Speed, Latency, and Quality
    2026-10-22 · lithosai.com · HN discussion
28. Max, Modular's inference and serving framework, is now source available
    2026-10-22 · modular.com · HN discussion
29. Show HN: The Daily Inference, a newspaper written and edited by AI agents
    2026-10-22 · dailyinference.news · HN discussion
30. Run LLMs Across Multiple Machines: Distributed Inference Guide for Your Homelab (2026)
    2026-10-22 · lucasaguiar.xyz · HN discussion

All trends