Ai2 has released Stage 1 checkpoints for its byte-level language models, keeping original model weights unchanged while providing pre-trained byte components. The release allows researchers to experiment with new architectures and train systems without re-running the initial stage.
Ai2 announced that its paper on retrofitting language models to operate over bytes has been accepted to Nature, alongside releasing new checkpoints that extend the Bolmo method to Qwen and Llama.
Ai2 has published a technical report titled 'Supercharging Olmo-core for Efficient and Scalable MoE Training,' focusing on scalable and efficient MoE model training.
Ai2 says Google Cloud reran Olmo 3’s 7B pretraining and mid-training on Cloud TPUs and matched the original run on held-out evaluations.
Ai2 says Marin’s latest hero run includes about 1.8 trillion tokens from its Dolma 3.5 corpus, and that Marin’s data-mixture work draws on Ai2’s Olmix recipes and Organize the Web curation methodology. Will Held says Marin’s 535B run was built on 25 trillion tokens from 152 datasets licensed for training.
At Climate Week NYC, Ai2 and GlobalFishWatch announced a partnership on how AI and AI agents are used in ocean monitoring and enforcement. Ai2 describes the effort as purpose-built, open, and accountable to the people who use it.
