news
EMO: Pretraining mixture of experts for emergent modularity
May 9, 2026
EMO is a new pretraining approach for mixture-of-experts models aimed at producing emergent modularity. It matters because modular expert routing can improve parameter efficiency and specialization in large models, though the excerpt provides no further performance details.
Source: huggingface.co