Mistral 7B
Model Overview¶
The Mistral-7B-Instruct-v0.1 Large Language Model (LLM) is a instruct fine-tuned version of the Mistral-7B-v0.1 generative text model using a variety of publicly available conversation datasets.
-
Paper: Mistral 7B
-
Model Source: mistralai/Mistral-7B-Instruct-v0.1
QPC Configurations¶
| Precision | SoCs / Tensor slicing | NSP-Cores (per SoC) | Full Batch Size | Chunking Prompt Length | Context Length (CL) | Generated URL | Download |
|---|---|---|---|---|---|---|---|
| MXFP6 | 4 | 16 | 16 | 128 | 4096 | https://dc00tk1pxen80.cloudfront.net/SDK1.19.6/mistralai/Mistral-7B-Instruct-v0.1/qpc_16cores_128pl_4096cl_16fbs_4devices_mxfp6_mxint8.tar.gz | Download |
| MXFP6 | 4 | 16 | 1 | 128 | 4096 | https://dc00tk1pxen80.cloudfront.net/SDK1.19.6/mistralai/Mistral-7B-Instruct-v0.1/qpc_16cores_128pl_4096cl_1fbs_4devices_mxfp6_mxint8.tar.gz | Download |
| MXFP6 | 4 | 16 | 8 | 128 | 4096 | https://dc00tk1pxen80.cloudfront.net/SDK1.19.6/mistralai/Mistral-7B-Instruct-v0.1/qpc_16cores_128pl_4096cl_8fbs_4devices_mxfp6_mxint8.tar.gz | Download |