DeepSpeed
Add DataStates-LLM: Asynchronous Checkpointing Engine Support
#7166

Merged

Add DataStates-LLM: Asynchronous Checkpointing Engine Support #7166

sfc-gh-truwase merged 10 commits into deepspeedai:master from DataStates:dev

mauryaavinash95 requested a review from

tjruwase 1 year ago

mauryaavinash95 requested a review from

tohtana 1 year ago

mauryaavinash95 requested a review from

jomayeri 1 year ago

mauryaavinash95 requested a review from

loadams 1 year ago

mauryaavinash95 requested a review from

GuanhuaWang 1 year ago

mauryaavinash95 requested a review from

hwchen2017 1 year ago

mauryaavinash95 changed the title ~~Add DataStates-LLM: Asynchronous Checkpointing Engine Support #5763~~ Add DataStates-LLM: Asynchronous Checkpointing Engine Support 1 year ago

tjruwase commented on 2025-03-25

mauryaavinash95 force pushed from 968f6ca5 to 1c701d7c 1 year ago

mauryaavinash95 requested a review from

tjruwase 1 year ago

tjruwase commented on 2025-03-29

mauryaavinash95 requested a review from

tjruwase 361 days ago

tjruwase commented on 2025-04-02

tjruwase approved these changes on 2025-04-02

mauryaavinash95 force pushed from 11dd8437 to 3a820715 347 days ago

mauryaavinash95 force pushed from 3a820715 to 84f067b6 347 days ago

mauryaavinash95 force pushed from 84f067b6 to 6160140e 347 days ago

mauryaavinash95 force pushed from 6160140e to 4651ec29 347 days ago

mauryaavinash95 requested a review from

tjruwase 347 days ago

mauryaavinash95 closed this 176 days ago

mauryaavinash95 force pushed from e2cf199c to 7d9a2f2b 176 days ago

mauryaavinash95 reopened this 176 days ago

mauryaavinash95 force pushed from aace707a to c196e62a 176 days ago

mauryaavinash95 closed this 176 days ago

mauryaavinash95 force pushed from c196e62a to 7d9a2f2b 176 days ago

mauryaavinash95 reopened this 176 days ago

Update datastates using decoupled checkpointing APIs (fix pre-commit)

0270a7b9

mauryaavinash95 force pushed from acc4dc31 to 0270a7b9 176 days ago

sfc-gh-truwase commented on 2025-10-11

sfc-gh-truwase commented on 2025-10-15

sfc-gh-truwase approved these changes on 2025-10-15

Add persistence when committing checkpoints

8db3a488

Import datastates checkpoint engine locally

3a84c230

Export DataStates engine from runtime/checkpoint_engine

9e261e1b

mauryaavinash95 force pushed from bd9ef2b6 to 9e261e1b 164 days ago

DataStates set commit_info=None after committing

0a6ff214

Merge branch 'master' into dev

55dfe7e0

Merge branch 'master' into dev

0ea44730

Fix datastates ImportError

867523fc

sfc-gh-truwase enabled auto-merge (squash) 159 days ago

Merge branch 'master' into dev

4188eba3

Merge branch 'master' into dev

d38bb0e8

sfc-gh-truwase merged d1e62ff2 into master 156 days ago

Reviewers

sfc-gh-truwase

tjruwase

tohtana

jomayeri

loadams

GuanhuaWang

hwchen2017

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed Add DataStates-LLM: Asynchronous Checkpointing Engine Support #7166 Merged

Add DataStates-LLM: Asynchronous Checkpointing Engine Support #7166

DeepSpeed
Add DataStates-LLM: Asynchronous Checkpointing Engine Support
#7166

Merged