mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-4bit_g64-HQQ
Text Generation
•
Updated
•
19
•
9
Mixtral HQQ Quantized Models
4-bit and 2-bit Mixtral models quantized using https://github.com/mobiusml/hqq
-
-
mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-2bit_g16_s128-HQQ
Text Generation • Updated • 17 • 9 -
mobiuslabsgmbh/Mixtral-8x7B-v0.1-hf-2bit_g16_s128-HQQ
Text Generation • Updated • 33 • 4 -
mobiuslabsgmbh/Mixtral-8x7B-v0.1-hf-4bit_g64-HQQ
Text Generation • Updated • 14 • 1
mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bit-HQQ
Text Generation • Updated • 25 • 38Note If you are considering 2-bit instruct model use this one.
mobiuslabsgmbh/Mixtral-8x7B-v0.1-hf-attn-4bit-moe-2bit-HQQ
Text Generation • Updated • 24 • 6Note If you are considering 2-bit base model use this one.
mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bit-metaoffload-HQQ
Text Generation • Updated • 35 • 15Note If you are considering 2-bit base model but is GPU pure this is a good option. Requires 13GB of RAM, but it will be slower.
-
mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-3bit-metaoffload-HQQ
Text Generation • Updated • 21 • 13 -
mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bitgs8-metaoffload-HQQ
Text Generation • Updated • 38 • 20