Show HN: Benchmarks of continual learning model trained for a month on a laptop

Category: ai-ml

Tags: continual-learning, language-model, mixture-of-experts, byte-level, low-vram-training

Score: 6.8/10 (Innovation: 7, Technical: 8, Documentation: 7, Utility: 5)

mini-AGI is a continual-learning byte-level language model that trains from scratch on a single 8GB GPU by paging expert weights between disk and VRAM, avoiding catastrophic forgetting through dynamic expert growth and pruning. It demonstrates that personal models can keep learning from a stream of data on consumer hardware without a frozen base.

Target audience: AI researchers and enthusiasts with consumer GPUs interested in lifelong learning and personal models

Repository: https://github.com/volotat/mini-AGI · Python · MIT · 1265 stars

View on Hacker News