The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) using a general purpose imatrix calibration dataset.
The imatrix is being used on the K-quants as well.

Layers	Context	Template
62	16384	<s>system {instructions} <s>human {prompt} <s>bot {response}<｜end▁of▁sentence｜>

Downloads last month: 10

GGUF

Model size

33.3B params

Architecture

llama

3-bit

4-bit

8-bit

View +2 files

Inference Providers NEW

Text Generation

This model is not currently available via any of the supported third-party Inference Providers, and the HF Inference API does not support gguf models with pipeline type text-generation