Baidu opens the ERNIE 5.5 Flash Base foundation model with 280 billion parameters and a context of up to one million tokens
Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .
The open base model will let researchers build their own reasoning systems and agents without paying for pre-training on 33.5 trillion tokens.
ERNIE 5.5 Flash Base is a sparse MoE model with 280 billion total parameters and 14 billion active parameters per token. It was pre-trained from scratch on 33.5 trillion tokens. Training used a context of up to 128 thousand tokens, while fine-tuning and inference support up to one million. Baidu released the base checkpoint before post-training so developers can perform their own domain adaptation, instruction tuning and reinforcement learning. To reduce the cost of long contexts, most layers compress every 128 tokens into one cache entry, while six layers retain detailed access to individual positions. In Baidu's own comparable evaluations, the model generally outperforms the open base versions of DeepSeek V4 Flash, Hy3 Preview and MiMo V2.5; these are the developer's results. Weights are available in FP8 and BF16 under a separate ERNIE licence.
Sources 2