|
--- |
|
library_name: transformers |
|
license: mit |
|
--- |
|
# RombUltima-32B |
|
|
|
**FINGU-AI/RombUltima-32B** is a merged model combining **rombodawg/Rombos-LLM-V2.5-Qwen-32b** and **Sakalti/ultiima-32B**. This model maintains the individual strengths of both Qwen and Ultima architectures while benefiting from an optimized fusion for improved reasoning, multilingual comprehension, and multi-turn conversation capabilities. |
|
|
|
--- |
|
|
|
## **Training & Fine-Tuning** |
|
RombUltima-32B is based on a **linear merge** of its parent models using equal weighting (0.5 each), resulting in a **balanced fusion** that leverages both structured knowledge from Rombos and enhanced generalization from Ultima. |
|
|
|
- **Tokenization Approach:** Uses a **union-based tokenizer** to maximize vocabulary coverage. |
|
- **Precision:** Trained and fine-tuned in **float16** for efficient inference. |
|
- **Long-Context Support:** Supports up to **32K tokens** (based on Qwen-32B), with stable generation up to **8K tokens**, depending on hardware constraints. |
|
- **Multilingual Strength:** Strong performance in **English, French, Chinese, and other global languages**. |
|
|
|
--- |
|
|
|
## **Performance & Benchmarks** |
|
### **OpenLLM Leaderboard** |
|
📌 **Coming Soon** – Evaluation against leading LLM benchmarks. |
|
|
|
### **MT-Bench** |
|
📌 **Coming Soon** – Multi-turn conversational performance analysis. |
|
|
|
--- |
|
|
|
## **Usage** |
|
You can run this model using the following code: |
|
|
|
```python |
|
import transformers |
|
from transformers import AutoTokenizer |
|
|
|
# Format prompt |
|
message = [ |
|
{"role": "system", "content": "You are a helpful AI assistant."}, |
|
{"role": "user", "content": "What is a Large Language Model?"} |
|
] |
|
tokenizer = AutoTokenizer.from_pretrained("FINGU-AI/RombUltima-32B") |
|
prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False) |
|
|
|
# Create pipeline |
|
pipeline = transformers.pipeline( |
|
"text-generation", |
|
model="FINGU-AI/RombUltima-32B", |
|
tokenizer=tokenizer |
|
) |
|
|
|
# Generate text |
|
sequences = pipeline( |
|
prompt, |
|
do_sample=True, |
|
temperature=0.7, |
|
top_p=0.9, |
|
num_return_sequences=1, |
|
max_length=200, |
|
) |
|
print(sequences[0]['generated_text']) |
|
``` |
|
|
|
--- |
|
|
|
## **Merging Details** |
|
- **Parent Models:** |
|
- 🟢 **rombodawg/Rombos-LLM-V2.5-Qwen-32b** (weight: 0.5) |
|
- 🟢 **Sakalti/ultiima-32B** (weight: 0.5) |
|
- **Merge Method:** Linear |
|
- **Tokenizer Source:** Union-based |
|
- **Precision:** Float16 |
|
|
|
--- |
|
|
|
## **Licensing & Intended Use** |
|
- **License:** Subject to original licenses of the merged models. |
|
- **Intended Use:** Research, content generation, multilingual applications, and general-purpose AI assistance. |
|
- **Limitations:** While the model excels in structured reasoning and multilingual understanding, **hallucinations and biases may still exist**. |
|
|
|
📌 **For feedback and contributions, visit: [FINGU-AI on Hugging Face](https://huggingface.co/FINGU-AI).** |
|
|
|
|
