Shanghai AI Lab and SJTU's LUMIA Lab release 8.9B open-weight model NCP-ArchPreview

Shanghai AI Lab and SJTU’s LUMIA Lab have released NCP-ArchPreview, an open-weight 8.9B language model under the Apache 2.0 license. The weights are in the NCP-ArchPreview collection, and a paper is also available.

The labs report that the model was trained on 5.73 trillion Dolma 3 tokens and reaches OLMo-3-7B’s final Stage 1 loss with 51.3% of the tokens, a 1.95× convergence gain. They report that its Stage 1 macro-average rises from 46.59 to 49.04, with gains of 5.99 on GSM8K and 4.28 on HumanEval.

NCP jointly predicts tokens and a concept sequence at one-quarter the length, then feeds those concepts back to guide generation. Domain adaptation updates only the 17 million-parameter concept module while the token backbone stays frozen. The labs report that concept-conditioned drafting improves mean accepted length by 4.17%, with negligible overhead.