All news

Baidu opens the ERNIE 5.5 Flash Base foundation model with 280 billion parameters and a context of up to one million tokens

Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .

The open base model will let researchers build their own reasoning systems and agents without paying for pre-training on 33.5 trillion tokens.

ERNIE 5.5 Flash Base is a sparse MoE model with 280 billion total parameters and 14 billion active parameters per token. It was pre-trained from scratch on 33.5 trillion tokens. Training used a context of up to 128 thousand tokens, while fine-tuning and inference support up to one million. Baidu released the base checkpoint before post-training so developers can perform their own domain adaptation, instruction tuning and reinforcement learning. To reduce the cost of long contexts, most layers compress every 128 tokens into one cache entry, while six layers retain detailed access to individual positions. In Baidu's own comparable evaluations, the model generally outperforms the open base versions of DeepSeek V4 Flash, Hy3 Preview and MiMo V2.5; these are the developer's results. Weights are available in FP8 and BF16 under a separate ERNIE licence.

History

Development of Baidu's ERNIE 5.5 Flash model

2 events
  1. Baidu introduces ERNIE Evolve, a system that runs its own experiments to improve SI
  2. Baidu opens the ERNIE 5.5 Flash Base foundation model with 280 billion parameters and a context of up to one million tokensOpen now