Qwen3 4B
Model Overview¶
Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support, with the following key features:
Uniquely support of seamless switching between thinking mode (for complex logical reasoning, math, and coding) and non-thinking mode (for efficient, general-purpose dialogue) within single model, ensuring optimal performance across various scenarios.
Significantly enhancement in its reasoning capabilities, surpassing previous QwQ (in thinking mode) and Qwen2.5 instruct models (in non-thinking mode) on mathematics, code generation, and commonsense logical reasoning.
Superior human preference alignment, excelling in creative writing, role-playing, multi-turn dialogues, and instruction following, to deliver a more natural, engaging, and immersive conversational experience.
Expertise in agent capabilities, enabling precise integration with external tools in both thinking and unthinking modes and achieving leading performance among open-source models in complex agent-based tasks.
Support of 100+ languages and dialects with strong capabilities for multilingual instruction following and translation.
- Qwen3-4B has the following features:
Type: Causal Language Models
Training Stage: Pretraining & Post-training
Number of Parameters: 4.0B
Number of Paramaters (Non-Embedding): 3.6B
Number of Layers: 36
Number of Attention Heads (GQA): 32 for Q and 8 for KV
Context Length: 32,768 natively and 131,072 tokens with YaRN. - Model Source: Qwen/Qwen3-4B
- License: apache-2.0
QPC Configurations¶
| Precision | SoCs / Tensor slicing | NSP-Cores (per SoC) | Full Batch Size | Chunking Prompt Length | Context Length (CL) | Generated URL | Download | Generation Date |
|---|---|---|---|---|---|---|---|---|
| MXFP6 | 1 | 16 | 1 | 128 | 4096 | https://dc00tk1pxen80.cloudfront.net/SDK1.20.4/Qwen/Qwen3-4B/Qwen3-4B_qpc_16cores_128pl_4096cl_1fbs_1devices_mxfp6_mxint8.tar.gz | Download | 05-Feb-2026 |
| MXFP6 | 2 | 16 | 1 | 128 | 4096 | https://dc00tk1pxen80.cloudfront.net/SDK1.20.4/Qwen/Qwen3-4B/Qwen3-4B_qpc_16cores_128pl_4096cl_1fbs_2devices_mxfp6_mxint8.tar.gz | Download | 05-Feb-2026 |
| MXFP6 | 1 | 16 | 1 | 128 | 8192 | https://dc00tk1pxen80.cloudfront.net/SDK1.20.4/Qwen/Qwen3-4B/Qwen3-4B_qpc_16cores_128pl_8192cl_1fbs_1devices_mxfp6_mxint8.tar.gz | Download | 05-Feb-2026 |
| MXFP6 | 2 | 16 | 1 | 128 | 8192 | https://dc00tk1pxen80.cloudfront.net/SDK1.20.4/Qwen/Qwen3-4B/Qwen3-4B_qpc_16cores_128pl_8192cl_1fbs_2devices_mxfp6_mxint8.tar.gz | Download | 05-Feb-2026 |
| MXFP6 | 4 | 16 | 1 | 128 | 4096 | https://dc00tk1pxen80.cloudfront.net/SDK1.20.4/Qwen/Qwen3-4B/Qwen3-4B_qpc_16cores_128pl_4096cl_1fbs_4devices_mxfp6_mxint8.tar.gz | Download | 26-Feb-2026 |
| MXFP6 | 4 | 16 | 1 | 128 | 8192 | https://dc00tk1pxen80.cloudfront.net/SDK1.20.4/Qwen/Qwen3-4B/Qwen3-4B_qpc_16cores_128pl_8192cl_1fbs_4devices_mxfp6_mxint8.tar.gz | Download | 26-Feb-2026 |
Run This Model¶
# Download QPC
mkdir -p Qwen/Qwen3-4B
cd Qwen/Qwen3-4B
wget <Download URL>
tar xzvf <downloaded filename.tar.gz>
# Run QPC
python3 -m QEfficient.cloud.execute --model_name Qwen/Qwen3-4B --qpc_path <path/to/qpc> --prompt "# shortest path algorithm\n" --generation_len 128
API Endpoint¶
# Start REST endpoint with vLLM
VLLM_QAIC_MAX_CPU_THREADS=8 VLLM_QAIC_QPC_PATH=/path/to/qpc python3 -m vllm.entrypoints.openai.api_server \
--host 0.0.0.0 \
--port 8000 \
--model Qwen/Qwen3-4B \
--max-model-len <Context Length> \
--max-num-seq <Full Batch Size> \
--max-seq_len-to-capture <Chunking Prompt Length> \
--device qaic \
--block-size 32