Update alignment heads in gen config
#3
by
sanchit-gandhi
- opened
Updates the alignment heads in the generation config as per the OpenAI recommendations. A quick inspection of the attention plots shows that these align relatively well with the token ids:
sanchit-gandhi
changed pull request title from
Upload config
to Update alignment heads in gen config
sanchit-gandhi
changed pull request status to
merged