cicdatopea
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -24,7 +24,7 @@ pip3 install git+https://github.com/intel/auto-round.git@bf16_inference
|
|
24 |
pip3 install auto-gptq
|
25 |
```
|
26 |
|
27 |
-
**This model is prone to overflow when running with
|
28 |
|
29 |
~~~python
|
30 |
from auto_round import AutoRoundConfig ##must import for autoround format
|
|
|
24 |
pip3 install auto-gptq
|
25 |
```
|
26 |
|
27 |
+
**This model is prone to overflow when running with int4 kernel with FP16 computation dtype** and does not support CPU, as it explicitly relies on CUDA operations in the model files. While we have implemented several workarounds to ensure functionality, **some prompts may still produce unexpected and random outputs**.
|
28 |
|
29 |
~~~python
|
30 |
from auto_round import AutoRoundConfig ##must import for autoround format
|