DownloadERNIE-4.5-300B-A47B-Base-Paddle is a powerful large language model by Baidu, based on a 300B parameter Mixture-of-Experts (MoE) architecture. It activates 47B parameters per token and is optimized for high-quality text generation and reasoning. This model is part of the ERNIE 4.5 series and leverages a heterogeneous MoE structure to balance performance and efficiency. It was trained in stages, starting with language understanding before expanding to include vision capabilities—though this variant focuses solely on text. Built using PaddlePaddle, it supports advanced infrastructure features like FP8 mixed-precision training, hybrid parallelism, and 4-bit/2-bit quantization for scalable deployment. The model supports long-context tasks with a maximum sequence length of 131,072 tokens. ERNIEKit enables easy fine-tuning using LoRA, SFT, or DPO, while FastDeploy and Transformers provide flexible deployment options across environments.
Features
- 300B total parameters, 47B activated per token
- Designed for high-performance text generation and reasoning
- Heterogeneous Mixture-of-Experts (MoE) structure
- Supports long-context processing (up to 131,072 tokens)
- Trained with FP8 mixed precision and advanced scheduling
- Compatible with ERNIEKit for SFT, LoRA, and DPO fine-tuning
- Deployable via FastDeploy and Transformers libraries
- Built on PaddlePaddle with multi-GPU and quantization support
Project Samples
