Show HN: TurboGPT: train 22KiB transformer in 13s

Category: ai-ml

Tags: cuda, gpt, transformer, c++, tiny-model

Score: 4.8/10 (Innovation: 5, Technical: 7, Documentation: 4, Utility: 3)

TurboGPT is a minimal byte-level GPT implementation in CUDA C++ designed for fast training on small datasets, achieving 2.53 BPB on the hn1g dataset after 1.5G tokens. It focuses on efficiency with a 22KiB model trained in 13 seconds, making it a niche tool for quick experimentation and educational purposes.

Target audience: machine learning engineers and researchers interested in efficient, low-level implementations

Repository: https://github.com/lostmsu/TurboGPT · C++ · MIT · 56 stars

View on Hacker News