DeepSpeed
Ulysses SP for HF Integration
#7268
Merged
Overview
Commits
39
Changes
View On GitHub

Ulysses SP for HF Integration #7268

stas00 merged 39 commits into master from ulysses
stas00
sfc-gh-sbekman add ds_grad_is_ready flag support
21025022
stas00 stas00 changed the title add ds_grad_is_ready flag support Ulysses SP for HF Integration 1 year ago
jeffra light sp mpu adaptation
95326f3d
sfc-gh-sbekman expand debug + revert bad ds PR that lead to mem leak
6284ea2a
sfc-gh-sbekman a lightweight version of MPU
12f4aa5a
sfc-gh-sbekman style
57341de4
sfc-gh-sbekman add all_gather_object
46b87ed3
sfc-gh-sbekman bulk of ulysses sp + features
358f0a7e
stas00 init file
5ff39a36
stas00 Merge remote-tracking branch 'origin/master' into ulysses
65f6a2e8
stas00 fix
a407bc77
stas00 get world_size correct for SP
aafffc53
stas00 improve
f6aad0c7
stas00 re-org
0907b809
stas00 cleanup
5d3960cd
stas00 cleanup
3c2a97b9
stas00 move
3cc5a86f
stas00 cleanup
babb59f0
stas00 refactor
2b1322f5
stas00 Merge remote-tracking branch 'origin/master' into ulysses
5ffd9292
stas00 fix
a07b5606
stas00 stas00 marked this pull request as ready for review 1 year ago
stas00 stas00 requested a review from tjruwase tjruwase 1 year ago
stas00 stas00 requested a review from tohtana tohtana 1 year ago
stas00 stas00 requested a review from GuanhuaWang GuanhuaWang 1 year ago
stas00 stas00 requested a review from loadams loadams 1 year ago
stas00 Merge remote-tracking branch 'origin/master' into ulysses
c6bdc974
stas00 add tests
46434a7c
stas00 make move_to_device more generic
15101cd4
stas00 improvements
b91b1abc
stas00 transformers requirements
cb343a68
stas00 don't use old attn version
fcb6396a
stas00 fix tiled mlp
17e2ae5e
stas00 Merge remote-tracking branch 'origin/master' into ulysses
a54e3a33
stas00 cleanup
15cf6a2a
stas00 Merge branch 'master' into ulysses
d2c0d2d3
stas00 install transformers anchor version last
91c75248
stas00 Merge branch 'ulysses' of https://github.com:/deepspeedai/DeepSpeed i…
cb705e08
sfc-gh-truwase
sfc-gh-truwase commented on 2025-05-29
sfc-gh-truwase
sfc-gh-truwase commented on 2025-05-29
sfc-gh-truwase
sfc-gh-truwase commented on 2025-05-29
sfc-gh-truwase
sfc-gh-truwase commented on 2025-05-29
stas00 Update deepspeed/runtime/sequence_parallel/ulysses_sp.py
3c50fa7c
stas00 copyright fix
0bc35f65
stas00 replay fix
611e11e3
stas00 Merge branch 'master' into ulysses
8b23721a
stas00 revert
884a3695
stas00 explanations
344f9860
sfc-gh-truwase
sfc-gh-truwase approved these changes on 2025-05-30
stas00 Merge branch 'ulysses' of https://github.com:/deepspeedai/DeepSpeed i…
1c8fd7e1
stas00 stas00 enabled auto-merge (squash) 1 year ago
stas00
stas00 stas00 merged 4d00b38a into master 1 year ago
stas00 stas00 deleted the ulysses branch 1 year ago
stas00
sfc-gh-truwase

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum