DownloadERNIE-4.5-0.3B-Base-PT is a compact, fully dense transformer model with 360 million parameters, optimized for general-purpose text generation tasks. It belongs to the ERNIE 4.5 series by Baidu and leverages advanced pretraining techniques without relying on a Mixture-of-Experts (MoE) structure. The model features 18 transformer layers, 16 attention heads, and a maximum context length of 131,072 tokens, offering strong language understanding for its size. It can be fine-tuned using ERNIEKit with support for SFT, LoRA, and DPO training methods, making it highly adaptable. Compatible with the Hugging Face Transformers library, the model can be easily used in Python for inference or deployed via FastDeploy. This variant emphasizes portability and accessibility, enabling fast deployment even on less powerful hardware. Ideal for developers seeking a smaller model for prototyping, educational use, or lightweight production tasks.
Features
- 360M parameters with 18 transformer layers
- Dense architecture (non-MoE) for streamlined inference
- 131,072 token context window
- Optimized for English and Chinese text generation
- Fine-tuning supported via ERNIEKit (SFT, DPO, LoRA)
- Hugging Face Transformers and FastDeploy compatibility
- Python API example included for easy use
- Apache 2.0 license with commercial-use permissions
Project Samples
