Ai2 says Marin’s latest hero run includes about 1.8T Dolma 3.5 tokens

Ai2 says Marin’s latest hero run includes about 1.8 trillion tokens from Ai2’s Dolma 3.5 corpus. Ai2 also says Marin’s data-mixture work draws on its Olmix recipes and the Organize the Web data-curation methodology.

Will Held says Marin’s 535B run was built on 25 trillion tokens from 152 datasets with licenses that permit training.