neuralmagic
/

granite-3.1-8b-instruct-FP8-dynamic

compressed-tensors MLA support requires fp8 activations and weights in group 'group_0',

#1 opened 23 days ago by