PR #4752 Universal Checkpoint for Sequence Parallelism

Universal Checkpoint for Sequence Parallelism #4752

mrwyattii merged 10 commits into master from universal_ckpt_sp

Extend universal checkpointing to support ds seq parallelism

6e6f8171

samadejacobs requested a review from

jeffra 2 years ago

samadejacobs requested a review from

tjruwase 2 years ago

samadejacobs requested a review from

mrwyattii 2 years ago

tjruwase commented on 2023-11-29

Remove debug statements

d8e48820

Remove debug statements

d0e2d6f7

tjruwase commented on 2023-12-01

Remove pipeline_parallel flag

30a876c0

Fix formatting

0626e7c6

samadejacobs requested a review from

awan-10 2 years ago

samadejacobs requested a review from

cmikeh2 2 years ago

samadejacobs requested a review from

arashb 2 years ago

formatting

db0ea555

Merge branch 'master' into universal_ckpt_sp

331ce3fc

tjruwase commented on 2023-12-11

tjruwase removed review request from

arashb 2 years ago

tjruwase removed review request from

jeffra 2 years ago

tjruwase removed review request from

cmikeh2 2 years ago

tjruwase removed review request from

awan-10 2 years ago

tjruwase requested a review from

lekurile 2 years ago

Update deepspeed_checkpoint.py

7d3f5b06

Merge branch 'master' into universal_ckpt_sp

027c49fe

tjruwase approved these changes on 2023-12-14

Merge branch 'master' into universal_ckpt_sp

7c0ceb14

mrwyattii approved these changes on 2023-12-15

mrwyattii merged 8998707a into master 2 years ago

mrwyattii deleted the universal_ckpt_sp branch 2 years ago

Reviewers

tjruwase

mrwyattii

lekurile

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed Universal Checkpoint for Sequence Parallelism #4752 Merged

Universal Checkpoint for Sequence Parallelism #4752

DeepSpeed
Universal Checkpoint for Sequence Parallelism
#4752

Merged