[whisper] static kv cache #31166
sanchit-gandhi
merged
76 commits into
huggingface:main
from
sanchit-gandhi:whisper-static-kv
make work with cache abstraction
738ed90f
correct for static cache
624fa74c
hacks for compile
f2124f86
make fast
9f02f7d0
fix
2d7102e5
fix pos ids
cd9ce9b7
generate
abad0b90
fix sdpa
248be4d3
fix sdpa cache pos
9ba0da91
fix fa2
4ea437a0
clean fa2
92f94f84
integrate cache into generate
7ea0d169
make style
b4478c19
copies
b6cb7395
more copies
57a219b7
update eager
2d917081
update sdpa
11e79a90
update fa2
27d520b5
simplify
f72224db
use cache pos
fcf024a2
always compute cross-cache for debug
3f48947c
avoid recompiles
7a5a5ebd
fix fix
2eba4473
fix fix fix
0bb8cb68
more fix
bfac7691
try encoder-decoder cache (too messy)
93c97c1f
revert encoder-decoder cache
05f12a3f
check cross-attn cache
c1060dfa
use enc-dec dataclass
6ee17ccc
use richer enc-dec dataclass
606417b6
clean-up
e13b38e5
revert static cache changes
5a54a01d
small fixes
3daa6ad4
revert to cpu flag
c244bcb4
fix copies
e0588df8
Merge branch 'main' into whisper-static-kv
5813aa3d
add static slow test
b879c574
past k/v docstring
86a46edd
more docstrings
d2094215
cache_position docstrings
0cba8280
add to docs
05e95dce
add enc-dec cache to docs
e5c83939
make style
959bae3b
fix after rebase
832e0b91
fix beam
34d78731
style
a321cd67
fix generation strategies
f825daf9
fix most decoder-only tests
e5c33dc4
style
216665a0
skip test
11a2791d
sanchit-gandhi
commented
on 2024-06-20
more clean up
004e94d0
small docstrings
23b7c229
Apply suggestions from code review
1a87b2bf
add todo
d629233c
only crop self-attn
8c0ce1a3
check cache in mixin
0f8b34f8
Merge remote-tracking branch 'origin/whisper-static-kv' into whisper-…
2d09a414
style
dba80a0f
fix re-compile after rebase
df31a150
move `is_updated` logic to enc-dec wrapper
cadd3dbe
revert back
78422156
Merge remote-tracking branch 'origin/whisper-static-kv' into whisper-…
5fcfdea6
revert cache back
79db195c
finalise design
6d3997f0
fix
6a377d12
fix fix
0093919c
sanchit-gandhi
commented
on 2024-06-25
style
ff57b4ce
Update src/transformers/cache_utils.py
2d4a2a88
deprecate
1860c315
Merge remote-tracking branch 'origin/whisper-static-kv' into whisper-…
d8e738f6
updates
24183cb2
final updates
2bad47ce
Merge branch 'main' into whisper-static-kv
89823f33
style
f0f81301
gante
approved these changes
on 2024-06-27
ArthurZucker
approved these changes
on 2024-06-28
Merge branch 'main' into whisper-static-kv
d8e8d64c
style
e25c8e1c
fxmarty
commented
on 2024-07-02
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub