Fix sparse attention for small block-sizes #1545
jeffra
merged
9 commits into
master
from
fix-sparse-attn
fixing the softmax masking when using triangular masking
f7ef4b5e
Merge branch 'master' of github.com:microsoft/DeepSpeed
dfb603fe
Merge branch 'master' of github.com:microsoft/DeepSpeed
c5ecf325
Merge branch 'master' of github.com:microsoft/DeepSpeed
426ecf73
Merge branch 'master' of github.com:microsoft/DeepSpeed
fde63105
fixing the sparse attention for low block-size
542b25c4
remove attn
d926bf75
Merge branch 'master' into fix-sparse-attn
544c0bc3
Merge branch 'master' into fix-sparse-attn
abfe900e
jeffra
approved these changes
on 2021-11-12
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub