Mistral 7B
Model Overview¶
The Mistral-7B-Instruct-v0.1 Large Language Model (LLM) is a instruct fine-tuned version of the Mistral-7B-v0.1 generative text model using a variety of publicly available conversation datasets.
-
Paper: Mistral 7B
-
Model Source: mistralai/Mistral-7B-Instruct-v0.1
QPC Configurations¶
| Precision | SoCs / Tensor slicing | NSP-Cores (per SoC) | Full Batch Size | Chunking Prompt Length | Context Length (CL) | Generated URL | Download |
|---|---|---|---|---|---|---|---|
| MXFP6 | 4 | 16 | 1 | 1000 | 1600 | https://dc00tk1pxen80.cloudfront.net/SDK1.18.4/Mistral-7B-Instruct-v0.1/qpc_16cores_1bs_1000pl_1600cl_-1mos_1fbs_4devices_mxfp6_mxint8.tar.gz | Download |
| MXFP6 | 4 | 16 | 4 | 1000 | 1600 | https://dc00tk1pxen80.cloudfront.net/SDK1.18.4/Mistral-7B-Instruct-v0.1/qpc_16cores_1bs_1000pl_1600cl_-1mos_4fbs_4devices_mxfp6_mxint8.tar.gz | Download |
| MXFP6 | 4 | 16 | 8 | 1000 | 1600 | https://dc00tk1pxen80.cloudfront.net/SDK1.18.4/Mistral-7B-Instruct-v0.1/qpc_16cores_1bs_1000pl_1600cl_-1mos_8fbs_4devices_mxfp6_mxint8.tar.gz | Download |
| MXFP6 | 4 | 16 | 16 | 256 | 512 | https://dc00tk1pxen80.cloudfront.net/SDK1.18.4/Mistral-7B-Instruct-v0.1/qpc_16cores_1bs_256pl_512cl_-1mos_16fbs_4devices_mxfp6_mxint8.tar.gz | Download |
| MXFP6 | 4 | 16 | 1 | 256 | 512 | https://dc00tk1pxen80.cloudfront.net/SDK1.18.4/Mistral-7B-Instruct-v0.1/qpc_16cores_1bs_256pl_512cl_-1mos_1fbs_4devices_mxfp6_mxint8.tar.gz | Download |
| MXFP6 | 4 | 16 | 1 | 7742 | 8192 | https://dc00tk1pxen80.cloudfront.net/SDK1.18.4/Mistral-7B-Instruct-v0.1/qpc_16cores_1bs_7742pl_8192cl_-1mos_1fbs_8devices_mxfp6_mxint8.tar.gz | Download |
| MXFP6 | 4 | 16 | 32 | 7742 | 8192 | https://dc00tk1pxen80.cloudfront.net/SDK1.18.4/Mistral-7B-Instruct-v0.1/qpc_16cores_1bs_7742pl_8192cl_-1mos_32fbs_8devices_mxfp6_mxint8.tar.gz | Download |
| MXFP6 | 4 | 16 | 16 | 128 | 4096 | https://dc00tk1pxen80.cloudfront.net/SDK1.18.4/Mistral-7B-Instruct-v0.1/qpc_16cores_1bs_128pl_4096cl_-1mos_16fbs_4devices_mxfp6_mxint8.tar.gz | Download |
| MXFP6 | 4 | 16 | 1 | 128 | 4096 | https://dc00tk1pxen80.cloudfront.net/SDK1.18.4/Mistral-7B-Instruct-v0.1/qpc_16cores_1bs_128pl_4096cl_-1mos_1fbs_4devices_mxfp6_mxint8.tar.gz | Download |
| MXFP6 | 4 | 16 | 8 | 128 | 4096 | https://dc00tk1pxen80.cloudfront.net/SDK1.18.4/Mistral-7B-Instruct-v0.1/qpc_16cores_1bs_128pl_4096cl_-1mos_8fbs_4devices_mxfp6_mxint8.tar.gz | Download |