2025 06 Pretraining Paper
A new paper A Minimalist Optimizer Design for LLM Pretraining, joint work with Thanos, Jiaxiang and Andi is available here. In this work, we propose an approach that builds efficient pretraining algorithms from scratch.
A new paper A Minimalist Optimizer Design for LLM Pretraining, joint work with Thanos, Jiaxiang and Andi is available here. In this work, we propose an approach that builds efficient pretraining algorithms from scratch.