DeepSpeed
[AutoTP] Fix ZeRO-3 checkpoint consolidation to gather across TP and DP
#8168
Merged
Overview
Commits
12
Changes
View On GitHub

[AutoTP] Fix ZeRO-3 checkpoint consolidation to gather across TP and DP #8168

tohtana merged 12 commits into deepspeedai:master from delock:gma/investigate-autotp-zero3-ucp
delock
delock [AutoTP] Fix ZeRO-3 checkpoint consolidation to gather across TP and DP
7b9dccb9
delock delock requested a review from tjruwase tjruwase 42 days ago
delock delock requested a review from tohtana tohtana 42 days ago
delock delock requested a review from loadams loadams 42 days ago
delock delock marked this pull request as draft 42 days ago
chatgpt-codex-connector
chatgpt-codex-connector commented on 2026-07-23
delock [AutoTP] Handle uneven TP shards in ZeRO-3 universal conversion
77f7204a
delock [AutoTP][docs] Update AutoTP docs to reflect ZeRO-3 support
7e1c059b
delock delock marked this pull request as ready for review 42 days ago
chatgpt-codex-connector
chatgpt-codex-connector commented on 2026-07-23
delock delock force pushed from 9a879b5d to 6a5f66d5 41 days ago
delock [AutoTP] Handle PP-local params in stage<=2 per-TP shapes
f3156b62
delock delock force pushed from 6a5f66d5 to f3156b62 41 days ago
delock delock marked this pull request as draft 41 days ago
delock delock force pushed from 9ff49d90 to ab2a220a 38 days ago
delock Merge branch 'master' into gma/investigate-autotp-zero3-ucp
4111888b
delock delock force pushed from ab2a220a to 4111888b 38 days ago
delock delock marked this pull request as ready for review 38 days ago
chatgpt-codex-connector
chatgpt-codex-connector commented on 2026-07-27
delock [AutoTP] Fix PP-major mp_rank grouping in stage<=2 universal conversion
2961fa32
delock Merge branch 'master' into gma/investigate-autotp-zero3-ucp
ed28c4c3
delock Merge branch 'master' into gma/investigate-autotp-zero3-ucp
ba92a4e8
tohtana
tohtana requested changes on 2026-07-30
delock [AutoTP] Classify AutoTP-unchanged params as TP-replicated in convers…
12fb28f4
delock [AutoTP] Remove obsolete test_autotp_zero3_training_blocked
2c72bea7
delock [AutoTP] Drop dead data_partition writes in TP gather_params
5519f395
delock [AutoTP][docs] Correct uneven-sharding claim in conversion limitation
dcc344cc
delock delock requested a review from hwchen2017 hwchen2017 34 days ago
delock
delock delock requested a review from tohtana tohtana 34 days ago
tohtana tohtana enabled auto-merge 31 days ago
tohtana
tohtana approved these changes on 2026-08-02
tohtana tohtana merged eec237ee into master 31 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum