Models · Updated 16 Oct, 06:32 pm IST
Transformers v5.20.0–v5.20.1 adds Muse Spark Open, Beam and MoE models
Why it matters for readers: If you follow AI models, these releases add notable new models that handle multiple media types and focus on coding/agent tasks.
- Muse Spark Open is a new multimodal mixture-of-experts model with 816B total parameters, a 1,048,576-token context window, and checkpoints in BF16 and NVFP4 formats.1
- Ling 3.0 Flash is added with native Transformers support as BailingHybridForCausalLM and a hybrid Mixture-of-Experts architecture that avoids trust_remote_code=True.1
- Kolibri 1 from Aleph Alpha is a MoE reasoning model routing tokens to many experts with a mix of sliding-window and full-attention layers.1
- Beam, announced in v5.20.1 by Reflection AI, is a token-efficient frontier-scale model aimed at coding and agentic use cases for teams to run on their own infrastructure.2
- The v5.20.1 release also fixes several generation and loading bugs, including PyTorch compatibility and MoE parallelism issues.2
Get a brief like this every morning
Uzha reads hundreds of sources and gives you the stories that matter for your work, with every source linked. Free.
Get startedHow this story developed
15 Oct, 11:35 pm · Transformers releases
Release v5.20.0
16 Oct, 06:32 pm · Transformers releases
Release v5.20.1