Show HN: Spectra – A drop-in optimizer that cuts LLM training VRAM by 50%

Category: ai-ml

Tags: optimizer, llm-training, vram-reduction, pytorch, frequency-domain

Score: 6.0/10 (Innovation: 6, Technical: 5, Documentation: 7, Utility: 6)

Spectra is a drop-in replacement for AdamW that reduces optimizer state memory by 50% by factorizing second-moment variance and filtering high-frequency noise via FFT. It claims faster training than GaLore without quantization collapse, making it appealing for memory-constrained LLM training.

Target audience: machine-learning engineers, researchers training large language models

Repository: https://github.com/SpectraSI/SpectraAdamw · Python · Apache-2.0 · 3 stars

View on Hacker News