transformers
Fix attention sink implementation in flex attention
#41083
Merged
Overview
Commits
13
Changes
View On GitHub

Fix attention sink implementation in flex attention #41083

SunMarc merged 13 commits into huggingface:main from SamuelBarryCS:fix-attention-sink
SamuelBarryCS
SamuelBarryCS Fix attention sink implementation in flex attention
ea9afc16
SamuelBarryCS fix dim
4c17ba6c
SamuelBarryCS fix
3e64462f
SamuelBarryCS Remove print
6f2e4c9a
SamuelBarryCS SamuelBarryCS marked this pull request as ready for review 244 days ago
SamuelBarryCS SamuelBarryCS changed the title [WIP] Fix attention sink in flex attention Fix attention sink implementation in flex attention 244 days ago
github-actions github-actions requested a review from MekkCyber MekkCyber 244 days ago
github-actions github-actions requested a review from SunMarc SunMarc 244 days ago
SamuelBarryCS
jonny-so
Cyrilvallez
SamuelBarryCS raisae error when return_lse is False yet s_aux is providewd
f37738fb
SamuelBarryCS
SamuelBarryCS Merge branch 'main' into fix-attention-sink
c94b1e27
SamuelBarryCS Clean test files for merge
cdf33258
SamuelBarryCS Merge branch 'fix-attention-sink' of github.com:SamuelBarryCS/transfo…
9c90c3d3
SamuelBarryCS
ArthurZucker
ArthurZucker approved these changes on 2025-09-24
ArthurZucker ArthurZucker added flex attention
SamuelBarryCS Update src/transformers/integrations/flex_attention.py
696bdd76
SamuelBarryCS force return lse
2e18a7ee
SamuelBarryCS Merge branch 'fix-attention-sink' of github.com:SamuelBarryCS/transfo…
f59dde4e
SamuelBarryCS Add to doc
58799e23
SamuelBarryCS
SamuelBarryCS Merge branch 'main' into fix-attention-sink
a2c147dc
SunMarc SunMarc enabled auto-merge (squash) 238 days ago
SunMarc
SunMarc approved these changes on 2025-09-29
SunMarc SunMarc merged 52cbc7c8 into main 238 days ago
HuggingFaceDocBuilderDev

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
flex attention
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum