Sarvam 1
Model Overview¶
Sarvam-1 is a 2-billion parameter language model specifically optimized for Indian languages. It provides best in-class performance in 10 Indic languages (bn, gu, hi, kn, ml, mr, or, pa, ta, te) when compared with popular models like Gemma-2-2B and Llama-3.2-3B. It is also competitive against the much larger models like Llama-3.1-8B in these languages. This is a text-completion model. It is meant to be finetuned on downstream tasks, and cannot be used directly as a chat or an instruction-following model.
Model Architecture¶
- Hidden size: 2048
- Intermediate size: 11,008
- Number of attention heads: 16
- Number of hidden layers: 28
- Number of key-value heads: 8
- Maximum position embeddings: 8,192
- Activation function: SwiGLU
- Positional embeddings: Rotary (RoPE) with theta=10,000
- Training: Grouped-query attention and bfloat16 mixed-precision
- Model Source: sarvamai/sarvam-1
- License: Non-commercial License.
QPC Configurations¶
| Precision | SoCs / Tensor slicing | NSP-Cores (per SoC) | Full Batch Size | Chunking Prompt Length | Context Length (CL) | Generated URL | Download | Generation Date |
|---|---|---|---|---|---|---|---|---|
| MXFP6 | 1 | 16 | 1 | 256 | 8192 | https://dc00tk1pxen80.cloudfront.net/SDK1.20.4/sarvamai/sarvam-1/sarvam-1_qpc_16cores_256pl_8192cl_1fbs_1devices_mxfp6_mxint8.tar.gz | Download | 27-Feb-2026 |
| MXFP6 | 4 | 16 | 1 | 256 | 8192 | https://dc00tk1pxen80.cloudfront.net/SDK1.20.4/sarvamai/sarvam-1/sarvam-1_qpc_16cores_256pl_8192cl_1fbs_4devices_mxfp6_mxint8.tar.gz | Download | 27-Feb-2026 |
Run This Model¶
# Download QPC
mkdir -p sarvamai/sarvam-1
cd sarvamai/sarvam-1
wget <Download URL>
tar xzvf <downloaded filename.tar.gz>
# Run QPC
python3 -m QEfficient.cloud.execute --model_name sarvamai/sarvam-1 --qpc_path <path/to/qpc> --prompt "# shortest path algorithm\n" --generation_len 128
API Endpoint¶
# Start REST endpoint with vLLM
VLLM_QAIC_MAX_CPU_THREADS=8 VLLM_QAIC_QPC_PATH=/path/to/qpc python3 -m vllm.entrypoints.openai.api_server \
--host 0.0.0.0 \
--port 8000 \
--model sarvamai/sarvam-1 \
--max-model-len <Context Length> \
--max-num-seq <Full Batch Size> \
--max-seq_len-to-capture <Chunking Prompt Length> \
--device qaic \
--block-size 32