DownloadERNIE-4.5-300B-A47B-PT is a post-trained, text-only Mixture-of-Experts (MoE) model with 300 billion total parameters and 47 billion active per token. Built on Baidu's ERNIE 4.5 architecture, it benefits from advanced innovations in pretraining and routing, including modality-isolated routing and token-balanced loss—even though this variant focuses purely on text. Designed for general-purpose natural language understanding and generation, it is fine-tuned using Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Unified Preference Optimization (UPO). Developers can deploy and fine-tune it using ERNIEKit or integrate it via Hugging Face Transformers with full support for custom prompts and chat templates. It supports highly efficient inference via FastDeploy, with multiple quantized variants (WINT4, WINT8, WINT2, FP8) for a range of hardware setups.
Features
- 300B total parameters with 47B activated per token
- Pure text-only MoE architecture optimized for LLM tasks
- Post-trained with SFT, DPO, and UPO methods
- Deployable with FastDeploy and Hugging Face Transformers
- Multiple quantization formats: FP8, WINT4/8/2
- Instruction fine-tuning and alignment with ERNIEKit
- Supports context lengths up to 131,072 tokens
- Includes prompt templates for web search in English and Chinese
Project Samples
