File size: 2,122 Bytes
c361859 5b71900 c361859 5b71900 c361859 c52e8b3 c361859 9029769 c361859 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
---
license: mit
train: false
inference: false
pipeline_tag: text-generation
---
This is a version of the <a href="https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B">DeepSeek-R1-Distill-Qwen-1.5B</a> model re-distilled for better performance.
## Performance
| Models | <a href="https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B">DeepSeek-R1-Distill-Qwen-1.5B</a> | <a href="https://huggingface.co/mobiuslabsgmbh/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1">DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1</a> |
|:-------------------:|:--------:|:----------------:|
| ARC (25-shot) | 40.96 | <b>41.3</b> |
| HellaSwag (10-shot)| 44 | <b>45.22</b> |
| MMLU (5-shot) | 39.27 | <b>42.01</b> |
| TruthfulQA-MC2 | 45.17 | <b>46.64</b> |
| Winogrande (5-shot)| 55.49 | <b>56.75</b> |
| GSM8K (5-shot) | 69.9 | <b>73.24</b> |
| Average | 49.13 | <b>50.86</b> |
| Models | <a href="https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B">DeepSeek-R1-Distill-Qwen-1.5B</a> | <a href="https://huggingface.co/mobiuslabsgmbh/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1">DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1</a> |
|:-------------------:|:--------:|:----------------:|
| GPQA (0-shot) | 26.96 | <b>27.8</b> |
| MMLU PRO (5-shot) | 16.74 | <b>19.44</b> |
| MUSR (0-shot) | 35.93 | <b>35.94</b> |
| BBH (3-shot) | 35.12 | 35.11 |
| Average | | |
## Usage
```Python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
compute_dtype = torch.bfloat16
device = 'cuda'
model_id = "mobiuslabsgmbh/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=compute_dtype, attn_implementation="sdpa", device_map=device)
tokenizer = AutoTokenizer.from_pretrained(model_id)
chat = tokenizer.apply_chat_template([{"role":"user", "content":"What is 1.5+102.2?"}], tokenize=True, add_generation_prompt=True, return_tensors="pt")
outputs = model_student.generate(chat.to(device), max_new_tokens=1024, do_sample=True)
print(tokenizer.decode(outputs[0]))
```
|