transformers
Fixes default value of `softmax_scale` in `PhiFlashAttention2`.
#28537

Merged

Fixes default value of `softmax_scale` in `PhiFlashAttention2`. #28537

ArthurZucker merged 2 commits into huggingface:main from fix-phi-tune

fix(phi): Phi does not use softmax_scale in Flash-Attention.

6a5ad209

chore(docs): Update Phi docs.

baf8d3ef

susnato commented on 2024-01-16

gugarosa marked this pull request as ready for review 2 years ago

ArthurZucker approved these changes on 2024-01-17

ArthurZucker merged d93ef7d7 into main 2 years ago

gugarosa deleted the fix-phi-tune branch 2 years ago

Reviewers

ArthurZucker

susnato

Assignees

No one assigned

Labels

None yet

Milestone

No milestone