transformers
Refactor flash attention implementation in transformers
#31446
Merged
Overview
Commits
62
Changes
View On GitHub

Refactor flash attention implementation in transformers #31446

fxmarty merged 62 commits into main from backend-compatible
ArthurZucker
ArthurZucker dumb commit
b66fdb0a
ArthurZucker nit
029ee11d
ArthurZucker Merge branch 'main' into backend-compatible
9a7885d6
ArthurZucker update
ac3e5b52
ArthurZucker something like this
a7c48bdf
ArthurZucker
ArthurZucker unpack in modeling utils
682f221d
ArthurZucker safe import
22011788
ArthurZucker oups
55a3503d
ArthurZucker update
b5cbaef5
ArthurZucker nits
7c6fdd7c
ArthurZucker diff convert gemma
08d7e1ed
ArthurZucker update
27044dad
ArthurZucker start propagating
ca316a06
ArthurZucker udpate other modeling code as well
ea932674
ArthurZucker update for sliding window models
4b67223a
ArthurZucker nits
d59ac0c5
ArthurZucker more init cleanups
a1d3866c
ArthurZucker styling
aea7f039
ArthurZucker fixup
f1bedd0a
ArthurZucker noice
86e2edce
ArthurZucker pass fixup
e90a9446
ArthurZucker typo typing_extension -> typing_extensions
093fbf5e
ArthurZucker torch.nn.functionnal -> torch.nn.functional
a1c56d2a
ArthurZucker add to import structure
1aad4a2d
ArthurZucker unpack
10bc1fae
ArthurZucker simplify a bit more for this first version
9f08ddb1
HuggingFaceDocBuilderDev
ArthurZucker nut
2e65e57b
ArthurZucker update
f8622e6c
ArthurZucker update
2bb43475
ArthurZucker nit
9be7579c
ArthurZucker ease the import of `Unpack`
889cbf8b
ArthurZucker remove useless `use_sliding_window`
070af2de
ArthurZucker no qua please
80057a0d
ArthurZucker protect import?
c0b024dd
ArthurZucker style
8f7d1c18
ArthurZucker ArthurZucker added run-slow
ArthurZucker [run-slow]
46b77f9f
ArthurZucker [run slow] llama,gemma,mistral,mixtral
4a98ee72
raghukiran1224
ArthurZucker
ArthurZucker ArthurZucker changed the title Ragged tensor compatible FlashAttention Ragged tensor compatible FlashAttention2 2 years ago
fxmarty
fxmarty commented on 2024-06-26
fxmarty
fxmarty commented on 2024-06-26
fxmarty
fxmarty commented on 2024-06-26
fxmarty
fxmarty commented on 2024-06-26
fxmarty
fxmarty commented on 2024-06-26
ArthurZucker ArthurZucker changed the title Ragged tensor compatible FlashAttention2 Refactor flash attention implementation in transformers 2 years ago
ArthurZucker remove extra kwargs
25b2c10d
ArthurZucker Merge branch 'main' of github.com:huggingface/transformers into backe…
8c3780d6
ArthurZucker fix llama
1d38daba
fxmarty
fxmarty commented on 2024-06-27
fxmarty
fxmarty commented on 2024-07-01
fxmarty address review comments
f64864a5
fxmarty apply diff_model_converter to modeling_gemma.py
565c5dce
fxmarty Merge branch 'main' into backend-compatible
2403ce53
fxmarty remove cache_position 1
c89571d0
fxmarty remove cache_position 2
32c2df85
fxmarty some cleaning
54a9fb04
fxmarty fxmarty requested a review from asartran asartran 2 years ago
fxmarty fxmarty marked this pull request as ready for review 2 years ago
fxmarty refactor gemma2 as well
206731ec
fxmarty Merge branch 'main' into backend-compatible
7c65fc7f
asartran
asartran approved these changes on 2024-07-02
fxmarty apply review comments
1be8c31e
fxmarty rename file to modeling_flash_attention_utils.py
8d181ea0
fxmarty Merge branch 'main' into backend-compatible
3a30cb68
fxmarty siglip refactor
c92028a1
fxmarty remove dead code
72439931
fxmarty is the hub down?
8b077d82
fxmarty still down?
a9796bc3
ArthurZucker
ArthurZucker commented on 2024-07-09
fxmarty fix siglip
6752a9c5
fxmarty
fxmarty Merge branch 'main' into backend-compatible
3a9cf1b2
fxmarty fix gemma2
b4d1df57
fxmarty fatal: Could not read from remote repository.
1e1bc2f3
fxmarty fix typo in softcap implem
c79ca837
fxmarty flacky
30dc1230
fxmarty Failed: Timeout >120.0s
fae68435
fxmarty fxmarty merged e3143952 into main 2 years ago
fxmarty fxmarty deleted the backend-compatible branch 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
run-slow
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum