Qwen2 is a 2024 generation of open-weight language models developed by the Qwen team. It includes dense and mixture-of-experts checkpoints in several sizes, but its context limits and licenses are not identical across the family.
Last verified: July 21, 2026.
Core Qwen2 checkpoints accept text and generate text. They should not be described as image, audio, or video models.
Qwen2 at a glance
- Announced in June 2024.
- Available as Base and Instruct checkpoints.
- Includes 0.5B, 1.5B, 7B, 57B-A14B, and 72B sizes.
- Context support varies by checkpoint.
- The 72B repositories do not use Apache 2.0.
- Core Qwen2 is a text-in, text-out model family.
Official Qwen2 model sizes
Each size below has a Base checkpoint and an instruction-tuned Instruct checkpoint. Use an Instruct model for chat and instruction-following applications. Base checkpoints are intended for continued training, evaluation, or specialized post-training.
| Model size | Architecture | Default or trained context | Documented extended context | Main repository license |
|---|---|---|---|---|
| 0.5B | Dense | 32,768 tokens | — | Apache 2.0 |
| 1.5B | Dense | 32,768 tokens | — | Apache 2.0 |
| 7B | Dense | 32,768 tokens | Up to 131,072 with YaRN | Apache 2.0 |
| 57B-A14B | Mixture of Experts | 32,768 tokens | Up to 65,536 with YaRN | Apache 2.0 |
| 72B | Dense | 32,768 tokens | Up to 131,072 with YaRN | Tongyi Qianwen License |
The labels 128K and 64K used in the Qwen2 release material describe supported long-context ranges for particular Instruct models. They are not one native context limit shared by every Qwen2 checkpoint.
Native context versus YaRN extension
The distributed configurations for the larger Qwen2 checkpoints use a 32,768-token original position limit. Reaching the longer documented ranges requires the official YaRN scaling configuration and a compatible inference framework.
- Qwen2-7B and Qwen2-72B use a YaRN factor of 4 for the documented 131,072-token range.
- Qwen2-57B-A14B uses a YaRN factor of 2 for the documented 65,536-token range.
- Longer context increases KV-cache and memory requirements.
- Static YaRN can reduce performance on shorter inputs, so it should be enabled only when long context is required.
Qwen2 licenses are model-specific
It is inaccurate to label the entire Qwen2 family as Apache 2.0. The principal 0.5B, 1.5B, 7B, and 57B-A14B repositories use Apache 2.0, while Qwen2-72B uses the custom Tongyi Qianwen License.
Review the license file in the exact Base, Instruct, or quantized repository you plan to use. The 72B license contains additional conditions, including a provision concerning commercial products or services above a stated monthly-active-user threshold. This page is not legal advice.
Is Qwen2 multimodal?
No. The core Qwen2 checkpoints listed above are causal language models that accept text and generate text.
The original Qwen-VL and Qwen-Audio families were published before Qwen2. They are not Qwen2 releases. Separate Qwen2-VL and Qwen2-Audio families were published after the core Qwen2 generation and have their own model IDs, architectures, inputs, and deployment requirements.
| Family | Relationship to core Qwen2 | Primary modality |
|---|---|---|
| Qwen-VL | Predates Qwen2 | Vision and text |
| Qwen-Audio | Predates Qwen2 | Audio and text |
| Qwen2 | Core generation | Text |
| Qwen2-Audio | Separate later family | Audio and text |
| Qwen2-VL | Separate later family | Vision and text |
Run Qwen2-7B-Instruct with Transformers
This example uses the official instruction-tuned 7B checkpoint and remains within its default context configuration. Actual memory requirements depend on precision, quantization, input length, output length, framework, and device placement.
pip install "transformers>=4.37.0" acceleratefrom transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Qwen/Qwen2-7B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype="auto",
device_map="auto",
)
messages = [
{
"role": "user",
"content": "Explain grouped-query attention in three concise points."
}
]
prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
inputs = tokenizer([prompt], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
new_tokens = outputs[0][inputs.input_ids.shape[-1]:]
answer = tokenizer.decode(new_tokens, skip_special_tokens=True)
print(answer)Qwen2 dense models require Transformers 4.37.0 or later. Qwen2 mixture-of-experts checkpoints require a framework version that includes Qwen2-MoE support.
Choosing a Qwen2 checkpoint
- Instruct: choose this for assistants, chat, summarization, and instruction-following.
- Base: choose this for continued pretraining, research, or custom post-training.
- 57B-A14B: a mixture-of-experts model; active parameters describe computation, not the complete storage size.
- 72B: review its custom license before deployment.
Qwen2 limitations
- Generated information can be inaccurate and should be verified.
- Core checkpoints do not process images, audio, or video.
- Long-context support does not guarantee perfect recall across the entire context.
- Extending context increases memory and latency.
- No single GPU or RAM figure applies to every precision and deployment configuration.
Frequently asked questions
Is every Qwen2 model licensed under Apache 2.0?
No. Qwen2-72B uses the Tongyi Qianwen License. Check the license in the exact repository before using a checkpoint.
Does every Qwen2 model support 128K context?
No. The limits vary by size. The 7B and 72B Instruct models document a 131,072-token YaRN configuration, the 57B-A14B model documents 65,536, and the smaller models document 32,768.
Are Qwen-VL and Qwen-Audio versions of Qwen2?
No. The original Qwen-VL and Qwen-Audio families predate Qwen2. Qwen2-VL and Qwen2-Audio are separate later families.
Official sources
- Qwen2 release announcement and model table
- Official Qwen2 model collection
- Qwen2-7B-Instruct model card
- Qwen2-57B-A14B-Instruct model card
- Qwen2-72B-Instruct model card
- Original Qwen-VL announcement
- Original Qwen-Audio repository
Continue with the Qwen2.5 model guide or review the later Qwen3 generation.
This independent reference is not operated by Alibaba Cloud or the Qwen team. Model names and trademarks belong to their respective owners.