transformers
Flash Attention 2 support for RoCm
#27611
Merged
Overview
Commits
45
Changes
View On GitHub

Flash Attention 2 support for RoCm #27611

fxmarty merged 45 commits into huggingface:main from fxmarty:flash-attn-2-support-rocm
fxmarty
fxmarty support FA2
639e22b0
fxmarty fix typo
100a464e
fxmarty fix broken tests
0279b14e
fxmarty fix more test errors
f449e86e
fxmarty left/right
cd076a1b
fxmarty fix bug
40d6a95b
fxmarty more test
dc192b75
fxmarty typo
2ed4492c
fxmarty fix layout flash attention falcon
0cf3fec7
fxmarty do not support this case
414dc77b
fxmarty use allclose instead of equal
333c3e60
fxmarty
fxmarty commented on 2023-11-20
fxmarty
fxmarty commented on 2023-11-20
HuggingFaceDocBuilderDev
fxmarty fix various bugs with flash attention
26f0455e
fxmarty bump
f0dd02c7
fxmarty fix test
278ef41f
fxmarty fix mistral
321bf4bd
fxmarty use skiptest instead of return that may be misleading
4712b483
fxmarty Merge branch 'fix-bugs-flash' into flash-attn-2-support-rocm
97b25f68
add fix causal arg flash attention
a6b1633e
ArthurZucker
ArthurZucker commented on 2023-11-21
fxmarty fix copies
669f545a
fxmarty more explicit comment
2df227ae
fxmarty still use self.is_causal
16b73d78
fxmarty fix causal argument
685c7ce6
fxmarty comment
9c8c6f75
fxmarty Merge branch 'main' into flash-attn-2-support-rocm
851343fc
fixes
7dfe765a
fxmarty update documentation
3ab0d309
fxmarty Merge branch 'flash-attn-2-support-rocm' of https://github.com/fxmart…
1f6ae8a4
fxmarty add link
bbc1f5aa
fxmarty wrong test
d0869c5c
fxmarty fxmarty marked this pull request as ready for review 2 years ago
fxmarty fxmarty requested a review from mfuntowicz mfuntowicz 2 years ago
fxmarty fxmarty requested a review from ArthurZucker ArthurZucker 2 years ago
fxmarty fxmarty requested a review from amyeroberts amyeroberts 2 years ago
fxmarty fxmarty requested a review from younesbelkada younesbelkada 2 years ago
simplify FA2 RoCm requirements
e76188c5
Merge branch 'flash-attn-2-support-rocm' of https://github.com/fxmart…
52e83f9e
Merge branch 'main' into flash-attn-2-support-rocm
ae7f4864
fxmarty update opt
e4a18852
mfuntowicz
mfuntowicz approved these changes on 2023-11-23
fxmarty
LysandreJik
LysandreJik approved these changes on 2023-11-24
younesbelkada
younesbelkada commented on 2023-11-24
fxmarty make flash_attn_uses_top_left_mask attribute private and precise comment
ce6e1473
fxmarty better error handling
5a603c09
ArthurZucker
ArthurZucker
ArthurZucker approved these changes on 2023-11-24
amyeroberts
amyeroberts approved these changes on 2023-11-24
younesbelkada
younesbelkada approved these changes on 2023-11-24
fxmarty Merge branch 'main' into flash-attn-2-support-rocm
c4809e6b
fix copy & mistral
6cf3bb7c
fxmarty Update src/transformers/modeling_utils.py
8ddbb298
fxmarty Update src/transformers/modeling_utils.py
55b36e42
fxmarty Update src/transformers/modeling_utils.py
9d0289b5
fxmarty Update src/transformers/utils/import_utils.py
60b3163c
fxmarty use is_flash_attn_greater_or_equal_2_10 instead of is_flash_attn_grea…
57025a4c
fxmarty fix merge
87ac9499
fxmarty simplify
733b9509
fxmarty inline args
3926b06d
fxmarty fxmarty merged 1da1302e into main 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum