Show HN: Fine-tuned 110M encoder beat 7B LLMs and hybrid search for NIST mapping

Category: ai-ml

Tags: cybersecurity, embedding-models, contrastive-learning, compliance, retrieval, fine-tuning

Score: 6.0/10 (Innovation: 6, Technical: 6, Documentation: 7, Utility: 5)

This project fine-tunes a compact 110M parameter encoder (BGE-Base) with contrastive learning to map CVE vulnerability descriptions to NIST SP 800-53 cybersecurity controls, achieving 77.45% top-1 accuracy versus <1% for zero-shot large models. It also empirically demonstrates that LLM reranking and BM25 hybrid search degrade performance in this domain, challenging common enterprise search assumptions. The repository provides the full pipeline: data, training scripts, evaluation harness, and model quantization.

Target audience: security engineers, ML engineers, compliance professionals

Repository: https://github.com/applied-inference-lab/autormf-ml-lab · Python · Apache-2.0 · 1 stars

View on Hacker News