LightningLM 0.1V — Reference training pipeline for the LightningLM family. 2B dense seed → 5B MoE → 9B MoE → 120B sparse MoE through TurboQuant-PreTraining on a single eight-GPU node. Companion code for *Reversible Foundations*.
pytorch language-models reproducibility quantized-training mixture-of-experts pretraining sparse-moe reversible-models brahmic-tokenizer lightninglm
-
Updated
Jun 4, 2026 - Python