File size: 2,872 Bytes
6781bef 9c70774 6781bef 98a732a 6781bef 98a732a 6781bef 98a732a 6781bef 98a732a 6781bef 98a732a 6781bef 98a732a 6781bef 98a732a 6781bef 98a732a 6781bef 98a732a 6781bef 98a732a 6781bef 98a732a 6781bef 98a732a 6781bef 98a732a 6781bef 98a732a 6781bef 98a732a 6781bef |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 |
---
library_name: transformers
license: mit
---
# RombUltima-32B
**FINGU-AI/RombUltima-32B** is a merged model combining **rombodawg/Rombos-LLM-V2.5-Qwen-32b** and **Sakalti/ultiima-32B**. This model maintains the individual strengths of both Qwen and Ultima architectures while benefiting from an optimized fusion for improved reasoning, multilingual comprehension, and multi-turn conversation capabilities.
---
## **Training & Fine-Tuning**
RombUltima-32B is based on a **linear merge** of its parent models using equal weighting (0.5 each), resulting in a **balanced fusion** that leverages both structured knowledge from Rombos and enhanced generalization from Ultima.
- **Tokenization Approach:** Uses a **union-based tokenizer** to maximize vocabulary coverage.
- **Precision:** Trained and fine-tuned in **float16** for efficient inference.
- **Long-Context Support:** Supports up to **32K tokens** (based on Qwen-32B), with stable generation up to **8K tokens**, depending on hardware constraints.
- **Multilingual Strength:** Strong performance in **English, French, Chinese, and other global languages**.
---
## **Performance & Benchmarks**
### **OpenLLM Leaderboard**
📌 **Coming Soon** – Evaluation against leading LLM benchmarks.
### **MT-Bench**
📌 **Coming Soon** – Multi-turn conversational performance analysis.
---
## **Usage**
You can run this model using the following code:
```python
import transformers
from transformers import AutoTokenizer
# Format prompt
message = [
{"role": "system", "content": "You are a helpful AI assistant."},
{"role": "user", "content": "What is a Large Language Model?"}
]
tokenizer = AutoTokenizer.from_pretrained("FINGU-AI/RombUltima-32B")
prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False)
# Create pipeline
pipeline = transformers.pipeline(
"text-generation",
model="FINGU-AI/RombUltima-32B",
tokenizer=tokenizer
)
# Generate text
sequences = pipeline(
prompt,
do_sample=True,
temperature=0.7,
top_p=0.9,
num_return_sequences=1,
max_length=200,
)
print(sequences[0]['generated_text'])
```
---
## **Merging Details**
- **Parent Models:**
- 🟢 **rombodawg/Rombos-LLM-V2.5-Qwen-32b** (weight: 0.5)
- 🟢 **Sakalti/ultiima-32B** (weight: 0.5)
- **Merge Method:** Linear
- **Tokenizer Source:** Union-based
- **Precision:** Float16
---
## **Licensing & Intended Use**
- **License:** Subject to original licenses of the merged models.
- **Intended Use:** Research, content generation, multilingual applications, and general-purpose AI assistance.
- **Limitations:** While the model excels in structured reasoning and multilingual understanding, **hallucinations and biases may still exist**.
📌 **For feedback and contributions, visit: [FINGU-AI on Hugging Face](https://huggingface.co/FINGU-AI).**
|