Show HN: Mini-AGI – Dynamic continual learning model trained on 8GB VRAM

	Show HN: Mini-AGI – Dynamic continual learning model trained on 8GB VRAM(github.com)
	274 points by volotat 23 days ago \| 73 comments
	tl;dr: Mini-AGI is a byte-level language model (~540M parameters) that trains from scratch on a single 8GB VRAM GPU by paging experts between disk, RAM, and VRAM, letting the parameter count scale with disk space. It uses a mixture-of-experts pool with adaptive depth (PonderNet-style) that grows and prunes itself during training, and claims to avoid catastrophic forgetting by running the shared trunk at 0.1x the experts' learning rate. Currently a toy-scale experiment demonstrating continual learning from a single data stream on consumer hardware, following a power law of L ∝ D^-0.239.
	HN Discussion: ↓Skeptical that the design actually avoids catastrophic forgetting as claimed ↓The 'Mini-AGI' name overpromises relative to what is delivered ↓Questions whether the model generalizes or merely memorizes, noting lack of benchmarks and coherent output ↑Appreciates the novel approach and effort, interested in exploring further •Suggests improvements like tokenization to enhance the architecture