DeepSpeed
[AutoEP] Run staged expert weight gradients with ZeRO-3
#9115
Open

[AutoEP] Run staged expert weight gradients with ZeRO-3 #9115

PKUWZP wants to merge 2 commits into autoep-deferred-dw-staging from autoep-deferred-dw-zero3
PKUWZP
PKUWZP PKUWZP force pushed from f51ffa97 to c76a7ea5 21 hours ago
PKUWZP PKUWZP force pushed from c76a7ea5 to b708bbfc 18 hours ago
PKUWZP PKUWZP marked this pull request as ready for review 18 hours ago
PKUWZP PKUWZP requested a review from tjruwase tjruwase 18 hours ago
PKUWZP PKUWZP requested a review from tohtana tohtana 18 hours ago
PKUWZP PKUWZP requested a review from hwchen2017 hwchen2017 18 hours ago
PKUWZP PKUWZP requested a review from loadams loadams 18 hours ago
chatgpt-codex-connector
chatgpt-codex-connector
chatgpt-codex-connector commented on 2026-11-18
PKUWZP PKUWZP force-pushed the autoep-deferred-dw-staging branch from ea50751b to 6ee0cbb9 15 hours ago
PKUWZP PKUWZP force pushed from b708bbfc to 16d50767 14 hours ago
PKUWZP [AutoEP] Run staged expert weight gradients with ZeRO-3
040996c3
PKUWZP [AutoEP] Do not keep gathered expert weights allocated until the back…
e27a91cc
PKUWZP PKUWZP force pushed from c381ff00 to e27a91cc 14 hours ago

Login to write a write a comment.

Login via GitHub

Assignees
No one assigned
Labels
Milestone