Accelerating Language Model Inference with Mixture of Attentions
•
24
Haitham Bou Ammar
hba123
AI & ML interests
LLMs, VLMs, Robotics, Reinforcement Learning, Bayesian Optimisation
Recent Activity
reacted
to
their
post
with 🚀
5 days ago
I have some New Year presents for you, #MachineLearning and #AI community! We just opened our code for new state-of-the-art results that beat EAGLE-2 and Medusa #LLM inference.
We also shared the model check pt on @huggingface! @MatthieuZ
Check the blog out: https://huggingface.co/blog/hba123/sotaspeculativedecoding
upvoted
an
article
5 days ago
Accelerating Language Model Inference with Mixture of Attentions
posted
an
update
5 days ago
I have some New Year presents for you, #MachineLearning and #AI community! We just opened our code for new state-of-the-art results that beat EAGLE-2 and Medusa #LLM inference.
We also shared the model check pt on @huggingface! @MatthieuZ
Check the blog out: https://huggingface.co/blog/hba123/sotaspeculativedecoding
Articles
Organizations
None yet
