Update alignment heads in gen config

#3
Whisper Distillation org
edited Apr 3, 2024

Updates the alignment heads in the generation config as per the OpenAI recommendations. A quick inspection of the attention plots shows that these align relatively well with the token ids:

Screenshot 2024-04-03 at 17.03.30.png

sanchit-gandhi changed pull request title from Upload config to Update alignment heads in gen config
sanchit-gandhi changed pull request status to merged
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment