Quantization
#1
by
shadow123
- opened
Can you make a GPTQ Int8 for it? Thx
Can you make a GPTQ Int8 for it? Thx
Maybe you are looking for this: https://huggingface.co/bartowski/huihui-ai_DeepSeek-R1-Distill-Llama-70B-abliterated-GGUF
The Q8_0 version takes 74.98GB, and the page also provide a model size range down to 16.75GB.