transformers
🔴[`Attention`] Attention refactor for Whisper-based models
#38235
Merged
Overview
Commits
36
Changes
View On GitHub

🔴[`Attention`] Attention refactor for Whisper-based models #38235

vasqu merged 36 commits into main from vas-whisper-attn-refactor
vasqu
vasqu start refactoring whisper
0d65fecd
vasqu revert for now
45cd987c
vasqu first step
91fb2ff2
vasqu carry over attn fixes
57dd18db
vasqu check if this works
04abc5d1
vasqu whisper has an off by one somewhere - cutting mask in any interface
6f8af141
HuggingFaceDocBuilderDev
vasqu make it based on interface
7c9bc047
vasqu remove some tests that were skipped but now work
bd0618f1
vasqu some fixes for whisper tests
c46a56f0
vasqu interface changes
8c9613b0
vasqu change the order of fix
1a05fe6c
vasqu some attention adjustments for eager + TP
d293c378
vasqu fix scaling
44e050bc
vasqu mask changes
2f3036a4
vasqu why does whisper contain those extra seq lens?
b6ab4290
vasqu fix from config for fa2 as input_ids is invalid
50b85506
vasqu fix another test
0288ef9f
vasqu another fix
21515337
vasqu disable flex attn due to compile issues
00da5dbe
vasqu copies and refactor for qwen audio since it somewhat relies on whisper
ba3092a9
vasqu fix scaling and smaller things
97444913
vasqu vasqu marked this pull request as ready for review 210 days ago
github-actions github-actions requested a review from ArthurZucker ArthurZucker 210 days ago
github-actions github-actions requested a review from eustlb eustlb 210 days ago
vasqu retrigger
0d5f01be
vasqu vasqu requested a review from gante gante 210 days ago
vasqu
vasqu commented on 2025-05-21
vasqu
vasqu commented on 2025-05-21
vasqu
vasqu commented on 2025-05-21
vasqu
vasqu commented on 2025-05-21
vasqu
vasqu commented on 2025-05-21
vasqu
vasqu commented on 2025-05-21
vasqu
vasqu commented on 2025-05-21
vasqu
vasqu commented on 2025-05-21
vasqu
vasqu commented on 2025-05-21
vasqu
vasqu commented on 2025-05-21
vasqu
vasqu commented on 2025-05-21
vasqu vasqu changed the title [`WIP`][`Attention`] Attention refactor for Whisper-based models [[`Attention`] Attention refactor for Whisper-based models 210 days ago
vasqu vasqu changed the title [[`Attention`] Attention refactor for Whisper-based models [`Attention`] Attention refactor for Whisper-based models 210 days ago
vasqu Merge branch 'main' into vas-whisper-attn-refactor
0c72d880
vasqu vasqu changed the title [`Attention`] Attention refactor for Whisper-based models 🔴[`Attention`] Attention refactor for Whisper-based models 210 days ago
gante
gante commented on 2025-05-21
vasqu
vasqu commented on 2025-05-21
vasqu new new interface version + more fixups
54ca390e
vasqu adjust qwen
7392d328
vasqu Merge branch 'main' into vas-whisper-attn-refactor
a12f2f43
vasqu
vasqu commented on 2025-05-22
vasqu add comment
2602b0f9
vasqu forgot this one
10666c60
vasqu change copies as whisper cuts on the mask
28951f0b
vasqu add guard
eeb5cf6f
vasqu vasqu requested a review from gante gante 209 days ago
vasqu
ArthurZucker
ArthurZucker approved these changes on 2025-05-23
vasqu
vasqu add flex attention
98836e6e
vasqu
Cyrilvallez
vasqu
vasqu switch to new mask function + add skips for torchscript
d4b0bd8b
gante
gante commented on 2025-05-23
gante
gante approved these changes on 2025-05-26
Cyrilvallez
vasqu remove old api with cache position
2d02cb19
vasqu Merge branch 'main' into vas-whisper-attn-refactor
395de624
Cyrilvallez
Cyrilvallez approved these changes on 2025-05-28
ArthurZucker
ArthurZucker approved these changes on 2025-05-28
vasqu last changes?
b6cf40c4
vasqu
vasqu trigger ci
338921c3
vasqu vasqu merged badc71b9 into main 203 days ago
vasqu vasqu deleted the vas-whisper-attn-refactor branch 203 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum