transformers
Fix sliding window attention used in Gemma2FlashAttention2
#32522

Merged

Fix sliding window attention used in Gemma2FlashAttention2 #32522

ArthurZucker merged 8 commits into huggingface:main from brcps12:fixing-sliding-window-attn

fix sliding window attention (flash2) in gemma2 model

13cb6c08

ArthurZucker commented on 2024-08-08

ArthurZucker commented on 2024-08-09

ArthurZucker added run-slow

[run-slow] gemma

e81fc78e

brcps12 requested a review from

ArthurZucker 1 year ago

ArthurZucker commented on 2024-08-12

fix slicing attention_mask for flash_attn2

f1adb8a7

fix slicing attention_mask when flash_attn is used

f912212d

Merge branch 'main' into fixing-sliding-window-attn

42f5d0e1

add missing comment

d3ae866a

slice the last seq_len tokens in the key, value states

73acbc18

revert code of slicing key, value states

991534ee

ArthurZucker approved these changes on 2024-08-12

ArthurZucker merged 342e3f9f into main 1 year ago

Reviewers

ArthurZucker

Assignees

No one assigned

Labels

run-slow

Milestone

No milestone