DownloadERNIE-4.5-300B-A47B-Base-PT is a post-trained variant of Baidu’s large-scale text-only MoE model, featuring 300 billion total parameters with 47 billion active per token. It builds upon the pretrained ERNIE 4.5 foundation and is optimized for natural language understanding and generation. The model supports advanced fine-tuning via SFT, LoRA, and DPO through the ERNIEKit training toolkit. It is compatible with PaddlePaddle and Transformers, making deployment and customization highly flexible. The architecture maintains scalability and efficiency using heterogeneous expert routing, FP8 precision, and quantized inference up to 2-bit. With a context length of 131,072 tokens, it’s designed for long-form generation and reasoning tasks. This post-trained version is ideal for developers seeking reliable LLM performance with high adaptability to real-world workloads.
Features
- 300B parameters with 47B activated per token
- Post-trained for improved language modeling tasks
- Supports LoRA, SFT, and DPO fine-tuning via ERNIEKit
- Long context support up to 131,072 tokens
- FP8 and quantized (4/2-bit) inference ready
- Built for PaddlePaddle and compatible with Transformers
- Supports vLLM serving with multi-GPU setups
- Optimized for instruction-following and dialogue tasks
Project Samples
