Hugging Face's logo

RichardErkhov
/
GritLM_-_GritLM-8x7B-KTO-gguf

GGUF
Inference Endpoints
conversational
Model card Files Community
RichardErkhov commited on
Commit
b08f9ad
·
verified ·
1 Parent(s): cdf2726

uploaded readme

Browse files
Files changed (1) hide show
  1. README.md +85 -0
README.md ADDED
@@ -0,0 +1,85 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Quantization made by Richard Erkhov.
2
+
3
+ [Github](https://github.com/RichardErkhov)
4
+
5
+ [Discord](https://discord.gg/pvy7H8DZMG)
6
+
7
+ [Request more models](https://github.com/RichardErkhov/quant_request)
8
+
9
+
10
+ GritLM-8x7B-KTO - GGUF
11
+ - Model creator: https://huggingface.co/GritLM/
12
+ - Original model: https://huggingface.co/GritLM/GritLM-8x7B-KTO/
13
+
14
+
15
+ | Name | Quant method | Size |
16
+ | ---- | ---- | ---- |
17
+ | [GritLM-8x7B-KTO.Q2_K.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q2_K.gguf) | Q2_K | 16.12GB |
18
+ | [GritLM-8x7B-KTO.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.IQ3_XS.gguf) | IQ3_XS | 18.02GB |
19
+ | [GritLM-8x7B-KTO.IQ3_S.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.IQ3_S.gguf) | IQ3_S | 19.03GB |
20
+ | [GritLM-8x7B-KTO.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q3_K_S.gguf) | Q3_K_S | 19.03GB |
21
+ | [GritLM-8x7B-KTO.IQ3_M.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.IQ3_M.gguf) | IQ3_M | 19.96GB |
22
+ | [GritLM-8x7B-KTO.Q3_K.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q3_K.gguf) | Q3_K | 21.0GB |
23
+ | [GritLM-8x7B-KTO.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q3_K_M.gguf) | Q3_K_M | 21.0GB |
24
+ | [GritLM-8x7B-KTO.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q3_K_L.gguf) | Q3_K_L | 22.51GB |
25
+ | [GritLM-8x7B-KTO.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.IQ4_XS.gguf) | IQ4_XS | 23.63GB |
26
+ | [GritLM-8x7B-KTO.Q4_0.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q4_0.gguf) | Q4_0 | 24.63GB |
27
+ | [GritLM-8x7B-KTO.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.IQ4_NL.gguf) | IQ4_NL | 24.91GB |
28
+ | [GritLM-8x7B-KTO.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q4_K_S.gguf) | Q4_K_S | 24.91GB |
29
+ | [GritLM-8x7B-KTO.Q4_K.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q4_K.gguf) | Q4_K | 26.49GB |
30
+ | [GritLM-8x7B-KTO.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q4_K_M.gguf) | Q4_K_M | 26.49GB |
31
+ | [GritLM-8x7B-KTO.Q4_1.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q4_1.gguf) | Q4_1 | 27.32GB |
32
+ | [GritLM-8x7B-KTO.Q5_0.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q5_0.gguf) | Q5_0 | 30.02GB |
33
+ | [GritLM-8x7B-KTO.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q5_K_S.gguf) | Q5_K_S | 30.02GB |
34
+ | [GritLM-8x7B-KTO.Q5_K.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q5_K.gguf) | Q5_K | 30.95GB |
35
+ | [GritLM-8x7B-KTO.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q5_K_M.gguf) | Q5_K_M | 30.95GB |
36
+ | [GritLM-8x7B-KTO.Q5_1.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q5_1.gguf) | Q5_1 | 32.71GB |
37
+ | [GritLM-8x7B-KTO.Q6_K.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/blob/main/GritLM-8x7B-KTO.Q6_K.gguf) | Q6_K | 35.74GB |
38
+ | [GritLM-8x7B-KTO.Q8_0.gguf](https://huggingface.co/RichardErkhov/GritLM_-_GritLM-8x7B-KTO-gguf/tree/main/) | Q8_0 | 46.22GB |
39
+
40
+
41
+
42
+
43
+ Original model description:
44
+ ---
45
+ pipeline_tag: text-generation
46
+ inference: true
47
+ license: apache-2.0
48
+ datasets:
49
+ - GritLM/tulu2
50
+
51
+ ---
52
+
53
+ # Model Summary
54
+
55
+ A [**KTO**](https://arxiv.org/abs/2402.01306) version of https://huggingface.co/GritLM/GritLM-8x7B
56
+
57
+ > GritLM is a generative representational instruction tuned language model. It unifies text representation (embedding) and text generation into a single model achieving state-of-the-art performance on both types of tasks.
58
+
59
+ - **Repository:** [ContextualAI/gritlm](https://github.com/ContextualAI/gritlm)
60
+ - **Paper:** https://arxiv.org/abs/2402.09906
61
+ - **Logs:** https://wandb.ai/muennighoff/gritlm/runs/0uui712t/overview
62
+ - **Script:** https://github.com/ContextualAI/gritlm/blob/main/scripts/training/train_gritlm_7b.sh
63
+
64
+ | Model | Description |
65
+ |-------|-------------|
66
+ | [GritLM 7B](https://hf.co/GritLM/GritLM-7B) | Mistral 7B finetuned using GRIT |
67
+ | [GritLM 8x7B](https://hf.co/GritLM/GritLM-8x7B) | Mixtral 8x7B finetuned using GRIT |
68
+
69
+ # Use
70
+
71
+ The model usage is documented [here](https://github.com/ContextualAI/gritlm?tab=readme-ov-file#inference).
72
+
73
+ # Citation
74
+
75
+ ```bibtex
76
+ @misc{muennighoff2024generative,
77
+ title={Generative Representational Instruction Tuning},
78
+ author={Niklas Muennighoff and Hongjin Su and Liang Wang and Nan Yang and Furu Wei and Tao Yu and Amanpreet Singh and Douwe Kiela},
79
+ year={2024},
80
+ eprint={2402.09906},
81
+ archivePrefix={arXiv},
82
+ primaryClass={cs.CL}
83
+ }
84
+ ```
85
+