Granite 3.2 8b instruct
Model Overview¶
Granite-3.2-8B-Instruct is an 8-billion-parameter, long-context AI model fine-tuned for thinking capabilities.
This model is designed to handle general instruction-following tasks and can be integrated into AI assistants across various domains, including business applications.
- Model Architecture: Granite-3.2-8B-Instruct is based on a decoder-only dense transformer architecture. Core components of this architecture are: GQA and RoPE, MLP with SwiGLU, RMSNorm, and shared input/output embeddings.
- Website: Granite Docs
- Model Source: ibm-granite/granite-3.2-8b-instruct
- Release Date: February 26th, 2025
- License: Apache 2.0
- Supported Languages: English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese.
QPC Configurations¶
| Precision | SoCs / Tensor slicing | NSP-Cores (per SoC) | Full Batch Size | Chunking Prompt Length | Context Length (CL) | Generated URL | Download |
|---|---|---|---|---|---|---|---|
| MXFP6 | 4 | 16 | 1 | 128 | 8192 | https://dc00tk1pxen80.cloudfront.net/SDK1.19.8/ibm-granite/granite-3.2-8b-instruct/qpc_16cores_128pl_8192cl_1fbs_4devices_mxfp6_mxint8.tar.gz | Download |
| MXFP6 | 4 | 16 | 8 | 128 | 8192 | https://dc00tk1pxen80.cloudfront.net/SDK1.19.8/ibm-granite/granite-3.2-8b-instruct/qpc_16cores_128pl_8192cl_8fbs_4devices_mxfp6_mxint8.tar.gz | Download |
| MXFP6 | 1 | 16 | 1 | 128 | 8192 | https://dc00tk1pxen80.cloudfront.net/SDK1.19.8/ibm-granite/granite-3.2-8b-instruct/qpc_16cores_128pl_8192cl_1fbs_1devices_mxfp6_mxint8.tar.gz | Download |