Show HN: DynamicTune – Closed-form trajectory weight surgery from 4B into 0.8B

Category: ai-ml

Tags: model-compression, knowledge-transfer, weight-surgery, mechanistic-interpretability, llm

Score: 7.3/10 (Innovation: 8, Technical: 8, Documentation: 7, Utility: 6)

DynamicTune is a zero-training weight surgery method that transfers capabilities from a large teacher model to a smaller student model by treating residual networks as dynamical systems. It uses closed-form linear algebra to project teacher velocity flows into student SwiGLU blocks, achieving significant gains on ARC-Challenge with minimal compute. The project introduces a spectral entropy-based layer selection strategy to avoid catastrophic interference in polysemantic layers.

Target audience: machine learning researchers, AI engineers, deep learning practitioners

Repository: https://github.com/dsadawq3/DynamicTune · Python · 9 stars

View on Hacker News