transformers
[`core` ] Integrate Flash attention 2 in most used models
#25598
Merged
Overview
Commits
99
Changes
View On GitHub

[`core` ] Integrate Flash attention 2 in most used models #25598

younesbelkada merged 99 commits into huggingface:main from younesbelkada:add-flash-attn-2
younesbelkada
younesbelkada v1
8bb77a1c
younesbelkada oops
2e18421a
younesbelkada working v1
fe5795ec
younesbelkada fixup
4bd15e2d
younesbelkada add some TODOs
49fe3185
younesbelkada Merge branch 'add-flash-attn-2' of https://github.com/younesbelkada/t…
f5d440b1
younesbelkada fixup
50491e83
younesbelkada younesbelkada changed the title [`PoC` / `core` ] Attempt to integration Flash attn 2 in most used models [`PoC` / `core` ] Attempt to integrate Flash attn 2 in most used models 2 years ago
fxmarty
HuggingFaceDocBuilderDev
sgugger
sgugger commented on 2023-08-21
pacman100
pacman100 commented on 2023-08-22
younesbelkada Merge remote-tracking branch 'upstream/main' into add-flash-attn-2
7df78c09
younesbelkada padding support + try with module replacement
0e30d13c
younesbelkada nit
ad8b9054
ydshieh
ydshieh commented on 2023-08-24
kai01ai
kai01ai commented on 2023-08-29
younesbelkada alternative design
3c31f10e
younesbelkada oops
2628bf3c
younesbelkada Merge remote-tracking branch 'upstream/main' into HEAD
56d0b49c
younesbelkada add `use_cache` support for llama
20d1b370
younesbelkada v1 falcon
a82f1ca4
younesbelkada nit
c72e8ff8
ArthurZucker
younesbelkada a bit of refactor
66823f92
younesbelkada nit
41f8f3d5
younesbelkada nits nits
a64a1a93
younesbelkada add v1 padding support falcon (even though it seemed to work before)
67e3fc29
younesbelkada nit
8444ab66
younesbelkada falcon works
8b1c2df2
younesbelkada fixup
c3ebcd2e
younesbelkada v1 tests
1c212d8d
younesbelkada nit
46187011
fxmarty fix generation llama flash
85ec9467
fxmarty Merge branch 'add-flash-attn-2' of https://github.com/younesbelkada/t…
2248f20d
fxmarty
fxmarty commented on 2023-09-01
younesbelkada update tests
0881ced1
younesbelkada Merge branch 'add-flash-attn-2' of https://github.com/younesbelkada/t…
a8a1b2df
younesbelkada fix tests + nits
2be3e03f
younesbelkada fix copies
b6d3e582
younesbelkada fix nit
b47e85c7
younesbelkada test- padding mask
db8bd640
younesbelkada stype
58848ab3
patrickvonplaten
patrickvonplaten commented on 2023-09-04
younesbelkada add more mem efficient support
3f73557d
fxmarty
younesbelkada Update src/transformers/modeling_utils.py
baae7363
younesbelkada fixup
55f61406
younesbelkada Merge remote-tracking branch 'upstream/main' into add-flash-attn-2
10d5c1b7
younesbelkada nit
3fb221a2
younesbelkada fixup
a931aeb9
younesbelkada remove it from config when saving
68a12040
younesbelkada fixup
36e0d6e0
flozi00
younesbelkada
younesbelkada
younesbelkada revert docstring
2beeb680
younesbelkada add more checks
7b5da2c5
younesbelkada use values
b99a582e
flozi00
younesbelkada oops
adaed459
younesbelkada
fxmarty
fxmarty new version
7f06af62
flozi00
younesbelkada fixup
2d36c6f1
younesbelkada Merge branch 'main' into add-flash-attn-2
a663fa4f
younesbelkada add same trick for falcon
9d3693f8
younesbelkada nit
65ae59c8
fxmarty
drisspg
drisspg
fxmarty
younesbelkada add another test
43185b5e
younesbelkada change tests
c61157e7
younesbelkada fix issues with GC and also falcon
2f177925
younesbelkada fixup
65c38617
younesbelkada oops
165a5030
younesbelkada younesbelkada requested a review from patrickvonplaten patrickvonplaten 2 years ago
younesbelkada younesbelkada requested a review from ArthurZucker ArthurZucker 2 years ago
younesbelkada younesbelkada requested a review from sgugger sgugger 2 years ago
younesbelkada
younesbelkada younesbelkada requested a review from pacman100 pacman100 2 years ago
younesbelkada
younesbelkada commented on 2023-09-12
younesbelkada
younesbelkada commented on 2023-09-12
ArthurZucker
ArthurZucker commented on 2023-09-12
fxmarty
younesbelkada Update src/transformers/models/falcon/modeling_falcon.py
5abc702f
younesbelkada add init_rope
5069e4a5
younesbelkada Merge branch 'add-flash-attn-2' of https://github.com/younesbelkada/t…
11400d89
younesbelkada updates
ace7939a
younesbelkada fix copies
fe9b16d8
younesbelkada Merge branch 'main' into add-flash-attn-2
6174c06f
younesbelkada fixup
acfc954f
younesbelkada fixup
33a0f629
younesbelkada more clarification
ee8ba206
younesbelkada fixup
e28fb0bf
younesbelkada right padding tests
025727c0
younesbelkada add docs
8f7e4008
younesbelkada add FA in docker image
3259392d
younesbelkada
younesbelkada commented on 2023-09-13
younesbelkada more clarifications
57a077b7
younesbelkada add some figures
e62b0b86
younesbelkada add todo
74194383
younesbelkada younesbelkada marked this pull request as ready for review 2 years ago
younesbelkada
younesbelkada commented on 2023-09-13
younesbelkada
younesbelkada commented on 2023-09-13
younesbelkada rectify comment
3ba5e983
patrickvonplaten
patrickvonplaten commented on 2023-09-14
patrickvonplaten
patrickvonplaten commented on 2023-09-14
younesbelkada Change to FA2
585e4634
younesbelkada younesbelkada requested a review from patrickvonplaten patrickvonplaten 2 years ago
younesbelkada younesbelkada requested a review from ArthurZucker ArthurZucker 2 years ago
younesbelkada younesbelkada requested a review from LysandreJik LysandreJik 2 years ago
ArthurZucker
ArthurZucker approved these changes on 2023-09-18
younesbelkada Update docs/source/en/perf_infer_gpu_one.md
ec0f8b9f
younesbelkada split in two lines
3e5ea353
younesbelkada change test name
4bb1bc53
younesbelkada Merge remote-tracking branch 'origin/main' into add-flash-attn-2
3ea46338
younesbelkada add more tests
b67c21e6
younesbelkada some clean up
5b735574
younesbelkada remove `rearrange` deps
48e3bcff
younesbelkada add more docs
04613840
younesbelkada younesbelkada changed the title [`PoC` / `core` ] Attempt to integrate Flash attn 2 in most used models [`core` ] Attempt to integrate Flash attn 2 in most used models 2 years ago
younesbelkada revert changes on dockerfile
8d72a66b
younesbelkada Revert "revert changes on dockerfile"
73b2f073
younesbelkada revert changes on dockerfile
fb7654cf
LysandreJik
LysandreJik commented on 2023-09-20
younesbelkada Apply suggestions from code review
a737bdea
younesbelkada address some comments
80951ae1
younesbelkada docs
6f7ff42b
younesbelkada use inheritance
257a6335
younesbelkada Update src/transformers/testing_utils.py
360da707
younesbelkada fixup
1d91bc4a
younesbelkada Merge branch 'main' into add-flash-attn-2
8ecab971
younesbelkada younesbelkada requested a review from LysandreJik LysandreJik 2 years ago
younesbelkada younesbelkada requested a review from ArthurZucker ArthurZucker 2 years ago
ArthurZucker
ArthurZucker commented on 2023-09-21
younesbelkada Apply suggestions from code review
7c5720f1
younesbelkada
younesbelkada commented on 2023-09-21
younesbelkada Update src/transformers/modeling_utils.py
28b82e20
younesbelkada final comments
84b57938
younesbelkada clean up
949172f1
younesbelkada style
825c7e0f
younesbelkada
LysandreJik
LysandreJik approved these changes on 2023-09-22
younesbelkada add cast + warning for PEFT models
1af232cf
younesbelkada fixup
d7f16c58
younesbelkada younesbelkada changed the title [`core` ] Attempt to integrate Flash attn 2 in most used models [`core` ] Integrate Flash attn 2 in most used models 2 years ago
younesbelkada younesbelkada changed the title [`core` ] Integrate Flash attn 2 in most used models [`core` ] Integrate Flash attention 2 in most used models 2 years ago
younesbelkada younesbelkada merged 368a58e6 into main 2 years ago
hiyouga
hiyouga commented on 2023-09-25
casper-hansen
congchan
congchan commented on 2023-10-12

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum