README.md

---
license: apache-2.0
base_model:
- Qwen/Qwen2.5-7B
pipeline_tag: text-generation
language:
- en
library_name: transformers
tags:
- text-generation-inference
---

## Model Description

Optimized Layer Merging (OLM)
Is a transformer optimization framework implementing automated layer recombination.

Olm create Frankenstein's monster out of language models by cherry-picking the best performing layers across different models to create a superior hybrid.
The core mechanism:

- Takes multiple language models as input
- Uses a base model as the foundation
- Iteratively replaces individual layers, evaluating performance on specified datasets
- Keeps the best performing layer at each position based on metrics like perplexity, exact match, and a custom "quality" score
- Builds a fusion model layer-by-layer while maintaining or improving performance

https://github.com/jeffmeloy/olm