Show HN: LoRA over GGUF – Train Qwen3.8-Flash-Next in 40G VRAM

Category: ai-ml

Tags: lora, gguf, low-vram-training, strix-halo, triton-kernels, moe

Score: 7.5/10 (Innovation: 8, Technical: 9, Documentation: 6, Utility: 7)

This project provides a recipe and custom kernels to train LoRA adapters on very large language models (up to 125B parameters) using GGUF-quantized base weights, enabling low-VRAM training on consumer hardware like Strix Halo. It combines transformers 5 with PEFT, custom Triton kernels (MMQ, GEMM, FlashAttention, MoE routing), and memory-saving techniques such as non-reentrant checkpointing and 8-bit AdamW to fit massive models in 40GB VRAM.

Target audience: AI/ML engineers, researchers, and hobbyists fine-tuning large language models on local hardware

Repository: https://github.com/woct0rdho/transformers5-qwen3.5-recipe · Python · Apache-2.0 · 22 stars

View on Hacker News