m-i commited on
Commit
b35b5e5
·
verified ·
1 Parent(s): a815c8b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +35 -0
README.md ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Ba2han/QwQenSeek-coder
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+ - mlx
8
+ - mlx-my-repo
9
+ ---
10
+
11
+ # m-i/QwQenSeek-coder-Q8-mlx
12
+
13
+ The Model [m-i/QwQenSeek-coder-Q8-mlx](https://huggingface.co/m-i/QwQenSeek-coder-Q8-mlx) was converted to MLX format from [Ba2han/QwQenSeek-coder](https://huggingface.co/Ba2han/QwQenSeek-coder) using mlx-lm version **0.20.5**.
14
+
15
+ ## Use with mlx
16
+
17
+ ```bash
18
+ pip install mlx-lm
19
+ ```
20
+
21
+ ```python
22
+ from mlx_lm import load, generate
23
+
24
+ model, tokenizer = load("m-i/QwQenSeek-coder-Q8-mlx")
25
+
26
+ prompt="hello"
27
+
28
+ if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
29
+ messages = [{"role": "user", "content": prompt}]
30
+ prompt = tokenizer.apply_chat_template(
31
+ messages, tokenize=False, add_generation_prompt=True
32
+ )
33
+
34
+ response = generate(model, tokenizer, prompt=prompt, verbose=True)
35
+ ```