Update README.md with text 2

README.md

@@ -10790,8 +10790,10 @@ license: mit
 <h4 align="center">
    <p>
        <a href="#acknowledgement">Acknowledgement</a>
-       <a href=#this-model>This Model</a>  |
+       <a href=#combination-of-embedding-models>Combination of Embedding Models</a>  |
        <a href=#usage>Usage</a>  |
+       <a href=#citation>Citation</a>  |
+       <a href=#license>License</a>
    <p>
 </h4>
 
@@ -10803,12 +10805,37 @@ Furthermore, we want to acknowledge the team of Marqo, who has worked on the ide
 
 ## Combination of Embedding Models
 
-- Embedding models become more powerful and applicabole in many use cases, but the next big challenge is to make them also more efficient in terms of resource consumption.
-- Our ambition is to experiment with the combination of two models to see if we can achieve a better performance with less resources. Early results have shown that models that are different to each others can complement each other and lead to better results. For coming up with a good combination, the selection of models is crucial, and the diversity (in terms of MTEB performance, architecture, training data, etc.) is a important part of it.
-- What kind of combination do we use? We have combined the embeddings of two models by concatenating them, the most straightforward technique of combination. Before concatenation, it is important to normalize the embeddings to make sure that the embeddings are in the same scale.
+### Overview
+Embedding models have become increasingly powerful and applicable across various use cases. However, the next significant challenge lies in enhancing their efficiency in terms of resource consumption. Our goal is to experiment with combining two embedding models to achieve better performance with fewer resources. 
 
-- We have combined the [Snowflake/snowflake-arctic-embed-m-v1.5](https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1.5) and [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) models to create this model. The combined model produces an embedding with 1152 dimensions (768+384) and has a total of 142M parameters (109+33).
-- This model combination performs well on the MTEB Leaderboard and is a good starting point for further experiments. However, we are aware that the combination of models is a complex topic and it is no good to only search for combinations that can climb the leaderboard. Yet, it is remarkable that the mere concatenation of two models can lead to an increase of Average nDCG@10 on the MTEB English Retrieval benchmark from 55.14 to 56.5, a climb of few spots in the leaderboard what is otherwise achieved with extensive efforts in engineering. Furthermore, it is interesting that the combination presented by the [Chimera model](https://huggingface.co/Marqo/marqo-chimera-arctic-bge-m) is performing significantly worse on the Leaderboard, even though the employed models are - on their own - more potent than pair of models combined in this repository. The reasons might be manifold, could it be the different number of model parameters, differences in the training process or maybe it just boils down to the fact how well two models can complement each other on the specific tasks of the respective benchmark. Anyways, we are looking forward to further experiments and discussions on this topic.
+### Key Insights
+1. **Diversity Matters**: Initial findings suggest that combining models with differing characteristics can complement each other, resulting in improved outcomes. To design an effective combination, the diversity of the models—evaluated by factors like MTEB performance, architecture, and training data—is crucial.
+2. **Combination Technique**:  
+   - We combine the embeddings of two models using the most straightforward approach: concatenation.
+   - Prior to concatenation, we normalize the embeddings to ensure they are on the same scale. This step is vital for achieving coherent and meaningful results.
+
+### Implementation
+We combined the following models:
+- **[Snowflake/snowflake-arctic-embed-m-v1.5](https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1.5)**  
+- **[BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5)**  
+
+#### Model Details
+- **Output Embedding Dimensions**: 1152 (768 + 384)  
+- **Total Parameters**: 142M (109M + 33M)
+
+### Results
+This combination demonstrated notable performance on the **MTEB Leaderboard**, offering a promising foundation for further experimentation:
+- **Performance Improvement**: The average nDCG@10 on the MTEB English Retrieval benchmark increased from **55.14 to 56.5**, climbing several spots on the leaderboard—a feat often requiring extensive engineering efforts.
+- **Comparison with Chimera Model**:  
+  Interestingly, the **[Chimera model](https://huggingface.co/Marqo/marqo-chimera-arctic-bge-m)**, which employs more potent models individually, performs worse on the leaderboard. This raises questions about:
+  - The role of parameter count.
+  - Differences in training processes.
+  - How effectively two models complement each other for specific benchmark tasks.
+
+### Future Directions
+While the results are promising, we acknowledge the complexity of model combinations and the importance of focusing on more than leaderboard rankings. The simplicity of concatenating embeddings yielding tangible gains emphasizes the potential for further exploration in this area.
+
+We look forward to conducting additional experiments and engaging in discussions to deepen our understanding of effective model combinations.
 
 ## Usage
 
@@ -10935,3 +10962,6 @@ mteb==1.12.94
   copyright = {Creative Commons Attribution 4.0 International}
 }
 ```
+
+## License
+Notice that Arctic M (v1.5) is licensed under [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0) and BGE Small (en; v1.5) is licensed under [MIT](https://opensource.org/licenses/MIT) license. Please refer to the licenses of the original models for more details.