Anthonyg5005 commited on
Commit
d1a44a7
·
verified ·
1 Parent(s): 7fb75e6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +87 -13
README.md CHANGED
@@ -1,13 +1,87 @@
1
- # Exl2 quants for [Llama-Guard-3-1B](https://huggingface.co/meta-llama/Llama-Guard-3-1B)
2
-
3
- ## Automatically quantized using the auto quant script from [hf-scripts](https://huggingface.co/anthonyg5005/hf-scripts)
4
-
5
- ### BPW:
6
-
7
- [4.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/4.0bpw)\
8
- [4.5](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/4.5bpw)\
9
- [5.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/5.0bpw)\
10
- [6.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/6.0bpw)\
11
- [6.5](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/6.5bpw)\
12
- [8.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/8.0bpw)\
13
- [measurement.json](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/blob/main/measurement.json)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ - de
5
+ - fr
6
+ - it
7
+ - pt
8
+ - hi
9
+ - es
10
+ - th
11
+ library_name: exllamav2
12
+ pipeline_tag: text-generation
13
+ tags:
14
+ - facebook
15
+ - meta
16
+ - pytorch
17
+ - llama
18
+ - llama-3
19
+ - safety
20
+ license: llama3.2
21
+ base_model:
22
+ - meta-llama/Llama-Guard-3-1B
23
+ ---
24
+ # Exl2 quants for [Llama-Guard-3-1B](https://huggingface.co/meta-llama/Llama-Guard-3-1B)
25
+
26
+ This is a quantized model from the llama guard series for use with [exllamav2](https://github.com/turboderp/exllamav2).\
27
+ Please refer to the original model card for instructions on how to use this model.
28
+
29
+ ### BPW:
30
+
31
+ [4.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/4.0bpw)\
32
+ [4.5](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/4.5bpw)\
33
+ [5.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/5.0bpw)\
34
+ [6.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/6.0bpw)\
35
+ [6.5](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/6.5bpw)\
36
+ [8.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/8.0bpw)\
37
+ [measurement.json](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/blob/main/measurement.json)
38
+
39
+ # How to download:
40
+
41
+ ### oobabooga's downloader
42
+
43
+ use something like [download-model.py](https://github.com/oobabooga/text-generation-webui/blob/main/download-model.py) to download with python requests.\
44
+ Install requirements:
45
+
46
+ ```shell
47
+ pip install requests tqdm
48
+ ```
49
+
50
+ Example for downloading 8bpw:
51
+
52
+ ```shell
53
+ python download-model.py Anthonyg5005/Llama-Guard-3-1B-exl2:8.0bpw
54
+ ```
55
+
56
+ ### huggingface-cli
57
+
58
+ You may also use huggingface-cli\
59
+ To install it, install python hf-hub
60
+
61
+ ```shell
62
+ pip install huggingface-hub
63
+ ```
64
+
65
+ Example for 8bpw:
66
+
67
+ ```shell
68
+ huggingface-cli download Anthonyg5005/Llama-Guard-3-1B-exl2 --local-dir Llama-Guard-3-1B-exl2-exl2-8bpw --revision 8.0bpw
69
+ ```
70
+ ### Git LFS (not recommended)
71
+
72
+ I would recommend the http downloaders over using git, they can resume downloads if failed and are much easier to work with.\
73
+ Make sure to have git and git LFS installed.\
74
+ Example for 8bpw download with git:
75
+
76
+ Have LFS file skip disabled
77
+ ```shell
78
+ # windows
79
+ set GIT_LFS_SKIP_SMUDGE=0
80
+ # linux
81
+ export GIT_LFS_SKIP_SMUDGE=0
82
+ ```
83
+
84
+ Clone repo branch
85
+ ```shell
86
+ git clone https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2 -b 8.0bpw
87
+ ```