THUDM

/

glm-4-9b-chat

like 668

Follow

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University 1.67k

包括 4bits 的

使用load in 4bit / 8bit可以使用，目前没有提供量化后的模型

We got autogptq 4bit models:

https://huggingface.co/LnL-AI/glm-4-9b-gptq-4bit-qubitium-r1
https://huggingface.co/LnL-AI/glm-4-9b-chat-gptq-4bit-qubitium-r1

@zRzRzRzRzRzRzR Please test our PRs that patches glm-4 with our. autogptq PR for quants.

4bit测试可以运行，8bit测试推理卡死了。gpu=v100