DownloadERNIE-4.5-300B-A47B-Paddle is a large-scale text-only Mixture-of-Experts (MoE) model built on Baidu’s ERNIE 4.5 architecture. With 300 billion total parameters and 47 billion activated per token, it is designed to handle complex natural language understanding and generation tasks. The model incorporates multimodal MoE pretraining infrastructure—although only the text modality is active in this version—leveraging innovations like modality-isolated routing, router orthogonal loss, and token-balanced optimization. It supports highly efficient deployment via PaddlePaddle, with quantization-ready configurations including 4-bit, 8-bit, and 2-bit variants for high-performance inference on large GPU clusters. Post-training techniques such as Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Unified Preference Optimization (UPO) have been applied for better alignment and response quality.
Features
- 300B total parameters with 47B active per token
- Optimized for high-quality text generation and comprehension
- Supports SFT, DPO, and UPO post-training strategies
- Built using PaddlePaddle with FastDeploy support
- Multiple quantization options (WINT4, WINT8, WINT2, FP8)
- Compatible with vLLM and Hugging Face Transformers
- Fine-tuning support through ERNIEKit with LoRA and multi-GPU options
- Handles long-context inputs up to 131,072 tokens
Project Samples
