Download
Qwen2-Audio is a large audio-language model by Alibaba Cloud, part of the Qwen series. It is trained to accept various audio signal inputs (including speech, sounds, etc.) and perform both voice chat and audio analysis, producing textual responses. It supports two major modes: Voice Chat (interactive voice only input) and Audio Analysis (audio + text instructions), with both base and instruction-tuned models. It is evaluated on many benchmarks (speech recognition, translation, sound classification, emotion, etc.), and offers pretrained models (e.g. 7B) released via ModelScope and Hugging Face. Code & examples provided with Hugging Face transformers, and usage via AutoProcessor, model classes etc. High performance on many standard benchmarks: ASR, speech-emotion recognition, vocal sound classification, speech translation etc.
Features
- Dual interaction modes: voice chat (audio only) and audio analysis (audio + text instruction)
- Includes both base model and instruction-tuned model (7B size)
- High performance on many standard benchmarks: ASR, speech-emotion recognition, vocal sound classification, speech translation etc.
- Code & examples provided with Hugging Face transformers, and usage via AutoProcessor, model classes etc.
- Supports audio inputs up to certain durations (audio clips under ~30 seconds perform best)
- Provides a web UI demo, evaluation scripts, and is released under open weights for research / usage
Project Samples
