Download
Qwen2.5-Omni is an end-to-end multimodal flagship model in the Qwen series by Alibaba Cloud, designed to process multiple modalities (text, images, audio, video) and generate responses both as text and natural speech in streaming real-time. It supports “Thinker-Talker” architecture, and introduces innovations for aligning modalities over time (for example synchronizing video/audio), robust speech generation, and low-VRAM/quantized versions to make usage more accessible. It holds state-of-the-art performance in many multimodal benchmarks, particularly spoken language understanding, audio reasoning, image/video understanding, etc. Very strong benchmark performance across modalities (audio understanding, speech recognition, image/video reasoning) and often outperforming or matching single-modality models at a similar scale. Real-time streaming responses, including natural speech synthesis (text-to-speech) and chunked inputs for low latency interaction.
Features
- Handles diverse input modalities: text, image, audio, video
- Real-time streaming responses, including natural speech synthesis (text-to-speech) and chunked inputs for low latency interaction
- Quantized model versions (4-bit GPTQ / AWQ) that reduce GPU memory needs by >50% while retaining comparable performance on multimodal evaluations
- Very strong benchmark performance across modalities (audio understanding, speech recognition, image/video reasoning) and often outperforming or matching single-modality models at similar scale
- Novel architectural elements like TMRoPE (Time-aligned Multimodal RoPE) to align timestamps between modalities like video and audio
- Cookbooks, examples, Docker / web demo support, low-VRAM mode, deployment via ModelScope, Hugging Face, etc.
Project Samples
