Qwen3-Omni is an open-weight multimodal model family developed by the Qwen team. It can understand text, images, audio, and video, but its outputs depend on the checkpoint you select. The Instruct checkpoint can return text and natural speech, while the Thinking and Captioner checkpoints return text.
Last verified: July 21, 2026.
Accuracy note: Qwen3-Omni does not generate images or videos. Image generation and editing belong to the separate Qwen-Image family.
Qwen3-Omni at a glance
| Checkpoint | Accepted input | Documented output | Best suited to |
|---|---|---|---|
Qwen/Qwen3-Omni-30B-A3B-Instruct | Text, images, audio, and video | Text and natural speech | General multimodal interaction and spoken responses |
Qwen/Qwen3-Omni-30B-A3B-Thinking | Text, images, audio, and video | Text | Multimodal tasks that benefit from explicit reasoning |
Qwen/Qwen3-Omni-30B-A3B-Captioner | Audio | Detailed text caption | Audio description and captioning |
These are separate downloadable checkpoints. The open-weight Instruct checkpoint does not become the Thinking checkpoint through an enable_thinking switch. Select the required model repository before loading the weights.
What can Qwen3-Omni do?
- Answer questions about images and visual documents.
- Analyze video content and relationships between visual and audio events.
- Transcribe, translate, describe, or discuss supported audio inputs.
- Combine text instructions with image, audio, or video evidence.
- Produce natural-speech responses with the Instruct checkpoint.
- Create detailed text descriptions of audio with the Captioner checkpoint.
The model’s ability to understand an image is different from generating a new image. Likewise, understanding or describing a video does not mean the model can render a new video file.
What does Qwen3-Omni not generate?
| Output type | Qwen3-Omni support | Correct model category |
|---|---|---|
| Text | Yes | Qwen3-Omni Instruct, Thinking, or Captioner |
| Natural speech | Instruct checkpoint only | Qwen3-Omni Instruct |
| Images | No | Qwen-Image |
| Video | No | A separately documented video-generation model |
| Music or general sound effects | Not documented as an output | A dedicated audio-generation model |
Context length: why two figures appear
Qwen’s technical report describes a final training stage using sequences up to 32,768 tokens. The distributed model configuration exposes 65,536 positions, and official serving examples use either 32,768 or 65,536 depending on the deployment setup.
These figures describe different parts of the published training and serving configuration. They do not support describing the open Qwen3-Omni checkpoints as having a native 128K or 1M context window.
Open weights, license, and deployment
The published Instruct, Thinking, and Captioner checkpoints use the Apache License 2.0. Qwen documents local use through Hugging Face Transformers, vLLM-Omni, Docker, Hugging Face downloads, and ModelScope downloads.
Qwen3-Omni is a large mixture-of-experts model. Hardware requirements depend on checkpoint, precision, media length, context allocation, and inference framework. It should not be presented as a lightweight mobile model or as guaranteed to run on a particular consumer GPU.
Official Transformers example
Install a Transformers version that includes the Qwen3-Omni implementation. The official repository recommends Transformers 5.2.0 or later and requires FFmpeg for media processing.
pip install "transformers>=5.2.0" accelerate
pip install -U qwen-omni-utils soundfile
import soundfile as sf
from transformers import (
Qwen3OmniMoeForConditionalGeneration,
Qwen3OmniMoeProcessor,
)
from qwen_omni_utils import process_mm_info
model_id = "Qwen/Qwen3-Omni-30B-A3B-Instruct"
model = Qwen3OmniMoeForConditionalGeneration.from_pretrained(
model_id,
dtype="auto",
device_map="auto",
)
processor = Qwen3OmniMoeProcessor.from_pretrained(model_id)
conversation = [
{
"role": "user",
"content": [
{
"type": "image",
"image": (
"https://qianwen-res.oss-cn-beijing.aliyuncs.com/"
"Qwen3-Omni/demo/cars.jpg"
),
},
{
"type": "audio",
"audio": (
"https://qianwen-res.oss-cn-beijing.aliyuncs.com/"
"Qwen3-Omni/demo/cough.wav"
),
},
{
"type": "text",
"text": "What can you see and hear? Answer in one short sentence.",
},
],
}
]
use_audio_in_video = True
prompt = processor.apply_chat_template(
conversation,
add_generation_prompt=True,
tokenize=False,
)
audios, images, videos = process_mm_info(
conversation,
use_audio_in_video=use_audio_in_video,
)
inputs = processor(
text=prompt,
audio=audios,
images=images,
videos=videos,
return_tensors="pt",
padding=True,
use_audio_in_video=use_audio_in_video,
)
inputs = inputs.to(model.device).to(model.dtype)
text_ids, audio = model.generate(
**inputs,
speaker="Ethan",
thinker_return_dict_in_generate=True,
use_audio_in_video=use_audio_in_video,
)
answer = processor.batch_decode(
text_ids.sequences[:, inputs["input_ids"].shape[1]:],
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)
print(answer[0])
if audio is not None:
sf.write(
"output.wav",
audio.reshape(-1).detach().cpu().numpy(),
samplerate=24000,
)
This example uses the Instruct checkpoint because that checkpoint includes the Talker component required for speech output. The Thinking checkpoint returns text instead.
Open checkpoints versus the hosted API
The Hugging Face identifiers above are not interchangeable with Alibaba Model Studio identifiers such as qwen3-omni-flash or qwen3-omni-flash-realtime. Hosted API modalities, request limits, thinking behavior, Function Calling, and speech output must be checked against the documentation for the selected API endpoint.
Qwen provides an audio Function Calling cookbook for the open family, while Alibaba documents Function Calling for its hosted HTTP Omni API. This should not be converted into a claim that every Qwen3-Omni checkpoint and serving framework supports the same tool-calling schema.
Practical limitations
- Longer or higher-resolution media increases memory and processing requirements.
- Generated descriptions, transcripts, and answers can contain errors.
- The model is not a medically validated diagnostic system.
- Visual understanding should not be described as verified biometric or identity recognition.
- Sensitive media should be processed only with appropriate consent, access controls, and retention policies.
Frequently asked questions
Can Qwen3-Omni generate images?
No. Qwen3-Omni can understand image input, but it does not generate image output. Use the Qwen-Image family for image generation or editing.
Can Qwen3-Omni generate video?
No. It can analyze video input, but video generation is not a documented output of the published Qwen3-Omni checkpoints.
Which checkpoint produces speech?
Qwen/Qwen3-Omni-30B-A3B-Instruct can produce text and natural speech. The Thinking and Captioner checkpoints produce text.
Is Qwen3-Omni open weight?
Yes. Qwen publishes the three listed checkpoints under the Apache License 2.0.
Official references
- Official Qwen3-Omni repository
- Qwen3-Omni Instruct model card
- Qwen3-Omni Thinking model card
- Qwen3-Omni Captioner model card
- Qwen3-Omni technical report
- Official Qwen-Image repository
This is an independent technical guide and is not affiliated with or endorsed by Alibaba Cloud or the Qwen team.