Show HN: Spectra-FFT – Cutting AdamW VRAM Usage by 50% for LLMs

Category: ai-ml

Tags: optimizer, llm-training, pytorch, vram-optimization, frequency-domain

Score: 4.5/10 (Innovation: 4, Technical: 5, Documentation: 5, Utility: 4)

Spectra-FFT is a PyTorch optimizer that claims to reduce AdamW's optimizer state memory by 50% using frequency-domain filtering and factorized second-moment variance. It positions itself as a faster, more stable alternative to GaLore and 8-bit Adam, but the project is very young with minimal community traction and unverified claims.

Target audience: AI/ML engineers and researchers training large language models on memory-constrained GPUs

Repository: https://www.spectralabs.si/ · Python · Apache-2.0 · 3 stars

View on Hacker News