Baidu’s ERNIE 4.5 family, a powerful foundation model series designed to enhance language understanding, reasoning and generation, has been officially opened-sourced. This release contains ten variants of models, ranging in size from 0.3B compact models to large Mixture-of-Experts architectures. The largest model has 424B parameters. Hugging Face has made these models freely available for the worldwide research and development community. This allows open experimentation, and gives a broader range of access to leading-edge Chinese language and multilingual technology.
Technical Overview of ERNIE 4.5 architecture
The ERNIE 4.5 model series is a continuation of Baidu’s previous iterations. This includes advanced models architectures that include both densely and sparsely-activated MoEs. MoE models are notable for their ability to scale parameter counts. The ERNIE 4.5 MoE-3B, and ERNIE 4.5 MoE-47B versions activate a small subset (usually 2 out of 64 experts) of experts. This allows them to keep the model’s expressivity and generalization abilities while keeping parameters under control.
The ERNIE models are developed using a combination of reinforcement learning and human feedback, supervised tuning (SFT), contrastive alignment (CA), and supervised fine-tuning techniques. Baidu’s multi-stage pipeline for pretraining has been used to train 5.6 trillion tokens from diverse domains, both in Chinese and English. The resulting models demonstrate high fidelity in instruction-following, multi-turn conversation, long-form generation, and reasoning benchmarks.
The Open-Source Model Release and Variations
The ERNIE 4.5 version includes ten different variants.
- Dense Models: ERNIE 4.5 0.3B, 0.5B, 1.8B, and 4B
- MoE Models: ERNIE 4.5MoE-3B (with varying parameters)
MoE 47B is a variant that activates just 3B parameters while it has a total 47B. Similarly, the 424B model—the largest ever released by Baidu—employs sparse activation strategies to make inference feasible and scalable. Both FP16 quantization and INT8 are supported by these models for an efficient deployment.
Performance Benchmarks
ERNIE 4.5 model shows notable improvements in key Chinese and Multilingual NLP tasks. Official technical report:
- You can find out more about this by clicking here. CMMLUERNIE 4.5 is a new version that surpasses the previous ERNIE releases and provides the highest accuracy for Chinese language comprehension.
- You can find out more about this by clicking here. MMLUThe benchmark ERNIE 4.547B, a multilingual LLM, is competitive with leading LLMs such as GPT-4 or Claude.
- The following are some of the ways to improve your own ability. long-form generationERNIE 4.5 has higher scores for coherence and facts when compared with Baidu’s own metrics.
In instruction-following tasks, the models benefit from contrastive fine-tuning, showing improved alignment with user intent and reduced hallucination rates compared to earlier ERNIE versions.

Application and deployment
ERNIE 4.5 are designed for a variety of different applications.
- The chatbot and assistant: Multilingual support and instruction-following alignment make it suitable for AI assistants.
- You can search and ask questions– High retrieval and production fidelity allows for integration RAG pipelines.
- Create Your Own ContentBetter factual foundations improve the quality of long-form texts and rich content.
- Code Multimodal ExtensionERNIE 4.5 supports multimodal extensions, even though the focus of this release is on text.
The ERNIE 4.5 Family can support up to 128K in context in certain variants. This is useful for tasks that require memory and reasoning over long documents.
You can also read our conclusion.
The ERNIE 4.5 Series represents a milestone in open-source AI research. This series offers a diverse set of models designed for scalable tasks, including multilingual ones, as well as those that follow instructions. Baidu has demonstrated its commitment to transparent and inclusive AI by releasing models that range from a lightweight model with 0.3B parameters to one with 424B parameters. ERNIE 4.5 will accelerate the global progress of natural language understanding. Its comprehensive documentation and open access to Hugging Face as well as its support for deployment is a key factor.
Take a look at the Paper You can also find out more about the following: Models on Hugging Face. This research is the work of researchers. Also, feel free to follow us on Twitter Don’t forget about our 100k+ ML SubReddit Subscribe Now our Newsletter.
Asif Razzaq, CEO of Marktechpost Media Inc. is a visionary engineer and entrepreneur who is dedicated to harnessing Artificial Intelligence’s potential for the social good. Marktechpost was his most recent venture. This platform, devoted to Artificial Intelligence, is known for the in-depth reporting of news on machine learning and deep understanding that is technically correct and understandable by all audiences. This platform has over 2,000,000 monthly views which shows its popularity.


