--- base_model: - Qwen/QwQ-32B-Preview - Qwen/Qwen2.5-32B-Instruct - deepseek-ai/DeepSeek-R1-Distill-Qwen-32B library_name: transformers tags: - mergekit - merge --- # **Qwen2.5-32B-DeepSeek-R1-Instruct** This model is a merged pre-trained language model created using MergeKit with the TIES merge method. It uses **Qwen/Qwen2.5-32B-Instruct** as the base and combines **Qwen/QwQ-32B-Preview** and **deepseek-ai/DeepSeek-R1-Distill-Qwen-32B** with equal weight and density. The merge configuration includes normalization, int8 masking, and `bfloat16` precision for optimized performance. # **Merge** This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). # **Merge Method** This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [Qwen/Qwen2.5-32B-Instruct](https://huggingface.co/Qwen/Qwen2.5-32B-Instruct) as a base. # **Models Merged** The following models were included in the merge: * [Qwen/QwQ-32B-Preview](https://huggingface.co/Qwen/QwQ-32B-Preview) * [deepseek-ai/DeepSeek-R1-Distill-Qwen-32B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B) # **Configuration** The following YAML configuration was used to produce this model: ```yaml models: - model: deepseek-ai/DeepSeek-R1-Distill-Qwen-32B - model: Qwen/QwQ-32B-Preview parameters: weight: 1 density: 1 merge_method: ties base_model: Qwen/Qwen2.5-32B-Instruct parameters: weight: 1 density: 1 normalize: true int8_mask: true dtype: bfloat16 ```