transformers
Fixes default value of `softmax_scale` in `PhiFlashAttention2`.
#28537
Merged

Fixes default value of `softmax_scale` in `PhiFlashAttention2`. #28537

ArthurZucker merged 2 commits into huggingface:main from fix-phi-tune
gugarosa
gugarosa fix(phi): Phi does not use softmax_scale in Flash-Attention.
6a5ad209
gugarosa chore(docs): Update Phi docs.
baf8d3ef
susnato
susnato commented on 2024-01-16
gugarosa gugarosa marked this pull request as ready for review 2 years ago
gugarosa
ArthurZucker
ArthurZucker approved these changes on 2024-01-17
ArthurZucker ArthurZucker merged d93ef7d7 into main 2 years ago
gugarosa
gugarosa gugarosa deleted the fix-phi-tune branch 2 years ago
younesbelkada
HuggingFaceDocBuilderDev

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
Milestone