Add Unified Sequence Parallel attention #12693
sayakpaul
merged
26 commits into
huggingface:main
from
Bissmella:unified-SP-attention
sayakpaul
commented
on 2025-12-11
initial scheme of unified-sp
123eb686
initial all_to_all_double
2a16bcc5
bug fixes, added cmnts
ec17a1af
unified attention prototype done
bbc65b80
remove raising value error in contextParallelConfig to enable unified…
32940ea9
bug fix
326c3dbb
feat: Adds Test for Unified SP Attention and Fixes a bug in Template …
93162212
bug fix, lse calculation, testing
3ac17e97
addressing comments
fd4c32b1
sequence parallelsim bug fixes
628f72d7
code format fixes
3a08cf4c
Apply style fixes
5579cf16
code formatting fix
da748347
added unified attention docs and removed test file
e681fe4b
Merge branch 'main' into unified-SP-attention
68f4fd51
Apply style fixes
5643bc93
tip for unified attention in docs at distributed_inference.md
27ec85f1
Merge branch 'main' into unified-SP-attention
9f345bd9
sayakpaul
commented
on 2025-12-29
Update distributed_inference.md, adding benchmarks
d990606b
Merge branch 'main' into unified-SP-attention
9b1407c7
sayakpaul
commented
on 2025-12-30
Update docs/source/en/training/distributed_inference.md
82320ad5
DN6
commented
on 2026-01-02
function name fix
6aa85f0f
DN6
commented
on 2026-01-02
fixed benchmark in docs
d1430d9b
Merge branch 'main' into unified-SP-attention
dc73b3ff
Merge branch 'main' into unified-SP-attention
54724b1b
DN6
approved these changes
on 2026-01-12
Merge branch 'main' into unified-SP-attention
eb6dec36
Login to write a write a comment.
Login via GitHub