Update README.md
Browse files
README.md
CHANGED
@@ -18,7 +18,7 @@ pipeline_tag: text-generation
|
|
18 |
## Merged below called "SISaAI"
|
19 |
- [Meta Llama](https://huggingface.co/NousResearch/Meta-Llama-3.1-8B-Instruct)
|
20 |
- [Korean Llama](https://huggingface.co/sh2orc/Llama-3.1-Korean-8B-Instruct)
|
21 |
-
- [Distill Llama](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B)
|
22 |
|
23 |
There is no such thing as a flawless system. It's about using it appropriately and reasonably without pushing it to its limits.
|
24 |
|
@@ -289,7 +289,32 @@ Third Eye Blind remains a beloved rock band with a dedicated fan base. Their mus
|
|
289 |
|
290 |
```
|
291 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
292 |
```
|
|
|
293 |
@article{Llama-3.1-SISaAI-Ko-merge-8B-Instruct,
|
294 |
title={asiansoul/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF},
|
295 |
author={Asiansoul called "Twp Eye Blind"},
|
|
|
18 |
## Merged below called "SISaAI"
|
19 |
- [Meta Llama](https://huggingface.co/NousResearch/Meta-Llama-3.1-8B-Instruct)
|
20 |
- [Korean Llama](https://huggingface.co/sh2orc/Llama-3.1-Korean-8B-Instruct)
|
21 |
+
- [DeepSeek R1 Distill Llama](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B)
|
22 |
|
23 |
There is no such thing as a flawless system. It's about using it appropriately and reasonably without pushing it to its limits.
|
24 |
|
|
|
289 |
|
290 |
```
|
291 |
|
292 |
+
## Provided Quants
|
293 |
+
|
294 |
+
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
295 |
+
|
296 |
+
| Link | Type | Size/GB | Notes |
|
297 |
+
|:-----|:-----|--------:|:------|
|
298 |
+
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF/resolve/main/Llama-3.1-SISaAI-Ko-merge-8B-Instruct.Q2_K.gguf) | Q2_K | 3.3 | |
|
299 |
+
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF/resolve/main/Llama-3.1-SISaAI-Ko-merge-8B-Instruct.Q3_K_S.gguf) | Q3_K_S | 3.8 | |
|
300 |
+
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF/resolve/main/Llama-3.1-SISaAI-Ko-merge-8B-Instruct.Q3_K_M.gguf) | Q3_K_M | 4.1 | lower quality |
|
301 |
+
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF/resolve/main/Llama-3.1-SISaAI-Ko-merge-8B-Instruct.Q3_K_L.gguf) | Q3_K_L | 4.4 | |
|
302 |
+
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF/resolve/main/Llama-3.1-SISaAI-Ko-merge-8B-Instruct.IQ4_XS.gguf) | IQ4_XS | 4.6 | |
|
303 |
+
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF/resolve/main/Llama-3.1-SISaAI-Ko-merge-8B-Instruct.Q4_K_S.gguf) | Q4_K_S | 4.8 | fast, recommended |
|
304 |
+
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF/resolve/main/Llama-3.1-SISaAI-Ko-merge-8B-Instruct.Q4_K_M.gguf) | Q4_K_M | 5.0 | fast, recommended |
|
305 |
+
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF/resolve/main/Llama-3.1-SISaAI-Ko-merge-8B-Instruct.Q5_K_S.gguf) | Q5_K_S | 5.7 | |
|
306 |
+
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF/resolve/main/Llama-3.1-SISaAI-Ko-merge-8B-Instruct.Q5_K_M.gguf) | Q5_K_M | 5.8 | |
|
307 |
+
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF/resolve/main/Llama-3.1-SISaAI-Ko-merge-8B-Instruct.Q6_K.gguf) | Q6_K | 6.7 | very good quality |
|
308 |
+
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF/resolve/main/Llama-3.1-SISaAI-Ko-merge-8B-Instruct.Q8_0.gguf) | Q8_0 | 8.6 | fast, best quality |
|
309 |
+
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF/resolve/main/Llama-3.1-SISaAI-Ko-merge-8B-Instruct.f16.gguf) | f16 | 16.2 | 16 bpw, overkill |
|
310 |
+
|
311 |
+
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
312 |
+
types (lower is better):
|
313 |
+
|
314 |
+
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
|
315 |
+
|
316 |
```
|
317 |
+
|
318 |
@article{Llama-3.1-SISaAI-Ko-merge-8B-Instruct,
|
319 |
title={asiansoul/Llama-3.1-SISaAI-Ko-merge-8B-Instruct-GGUF},
|
320 |
author={Asiansoul called "Twp Eye Blind"},
|