Update README.md

README.md

@@ -9,38 +9,42 @@ base_model:
 ---
 
 # Mistral-Small-3-Reasoner-s1
-A simple [Mistral-Small-24B-Instruct-2501](https://huggingface.co/mistralai/Mistral-Small-24B-Instruct-2501) finetune on the [s1K reasoning dataset by Muennighoff et al.](https://huggingface.co/datasets/simplescaling/s1K) to give the original model reasoning capabilities. Surprisingly, they appear to work even outside math/STEM subjects.
+A simple [Mistral-Small-24B-Instruct-2501](https://huggingface.co/mistralai/Mistral-Small-24B-Instruct-2501) finetune on the [s1K reasoning dataset by Muennighoff et al.](https://huggingface.co/datasets/simplescaling/s1K) to give the original model basic reasoning capabilities similar to DeepSeek R1's. Surprisingly, they appear to work even outside math/STEM subjects.
 
 ## Usage notes
 Prepend the assistant response with `<think>` to make the model engage in a chain-of-thought. This should happen automatically with math questions on an empty context, but it needs to be forced in longer conversations. When done thinking, the model will generate `</think>` and then the final response.
 
 Make sure that the models' output length is long enough, and be prepared to make the model continue its response if it stops prematurely.
 
+Low-depth instructions (perhaps at depth-0, just before the assistant's rsponse) can be beneficial in steering how the model should think. An additional `[SYSTEM_PROMPT]` can be used there.
+
+I advise to remove old chain-of-thoughts from the chat history before the model generates a new one, but I haven't tested this in depth.
+
 ## Prompting format
 ```
 [INST]User question.[/INST]<think>
-Chain of thought
+Chain of thought.
 </think>
 
 Model response.</s>
 ```
 
+## Known quirks and issues
+- Very long chain-of-thoughts might confuse the model during multi-turn conversations.
+- The model can override the requested formatting/style with that of the finetuning data.
+- Some of the non-reasoning capabilities of the original `Mistral-Small-24B-Instruct-2501` model might have degraded.
+- Most default guardrails apparently still work, but can be bypassed with a suitable prompt as with the original model.
+
 # What's in this repository
 - Checkpoints for epochs 1~5
 - LoRA adapter for the final model
-- Q4_K and Q6_K GGUF quantizations
-
-## Known quirks
-- The model can ramble indefinitely with open-ended questions at the beginning of the conversation.
-- The model can override the requested formatting/style with that of the finetuning data.
-- Some of the non-reasoning capabilities of the original `Mistral-Small-24B-Instruct-2501` model might have degraded.
-- Most model default guardrails still work, but can be bypassed with a suitable prompt as with the original model.
+- Some static GGUF quantizations
 
 ## Dataset
-Almost the entirety of the [s1K dataset](https://huggingface.co/datasets/simplescaling/s1K) was used with minimal modifications to make it properly work with Mistral-Small-3-Instruct, with the exception of 4 rows that didn't fit within the training sequence length of 8192 tokens and 16 of the shortest ones that have been used as the test set instead. No samples have been clipped and no system prompt was added. All samples are single-turn.
+Almost the entirety of the [s1K dataset](https://huggingface.co/datasets/simplescaling/s1K) was used with minimal modifications to make it properly work with Mistral-Small-3-Instruct, with the exception of 4 rows that didn't fit within the training sequence length of 8192 tokens and 16 of the shortest ones that have been used as the test set instead. No samples have been clipped and no system prompt was added. All were single-turn.
 
 ## Training hyperparameters
-I tried to more or less follow the indications in [appendix C in the paper](https://arxiv.org/abs/2501.19393) with the exception of using 4-bit LoRA finetuning and tuning the learning reate accordingly. The loss was not computed on questions within `[INST]...[/INST]` tags (including the tags), just on reasoning traces and solutions. The training sequence length was about the maximum I could use on one NVidia RTX3090 24GB GPU.
+I tried to roughly follow indications from [appendix C in the paper](https://arxiv.org/abs/2501.19393) with the notable exception of using 4-bit LoRA finetuning and tuning the learning rate accordingly. The loss was not computed on questions within `[INST]...[/INST]` tags (including the tags), just on reasoning traces and solutions. The training sequence length was about the maximum I could use on one NVidia RTX3090 24GB GPU. The total training time was about 18 hours.
 
 Overfitting (increasing eval loss) occurred after 1 epoch, but the training loss behavior was similar to that observed in the paper.