DeepSpeed
[AutoEP] Run staged expert weight gradients with ZeRO-3
#9115
Open

[AutoEP] Run staged expert weight gradients with ZeRO-3 #9115

PKUWZP wants to merge 1 commit into autoep-deferred-dw-staging from autoep-deferred-dw-zero3
PKUWZP
PKUWZP [AutoEP] Run staged expert weight gradients with ZeRO-3
f51ffa97

Login to write a write a comment.

Login via GitHub

Reviewers
No reviews
Assignees
No one assigned
Labels
Milestone