Show HN: We bought terafabs.si so Elon couldn't and made an arcade game

Category: ai-ml

Tags: optimizer, pytorch, memory-efficiency, llm-training, frequency-domain

Score: 5.8/10 (Innovation: 6, Technical: 6, Documentation: 6, Utility: 5)

Spectra is a PyTorch optimizer that reduces VRAM usage by factorizing second-moment variance and applying frequency-domain filtering to gradients. It claims a 50% memory reduction over AdamW with faster training than GaLore and Adafactor, though the benchmark results seem overly optimistic and warrant further validation.

Target audience: machine learning engineers, AI researchers

Repository: https://www.terafabs.si/ · Python · Apache-2.0 · 3 stars

View on Hacker News