DeepSpeed
re-introduce: stage3: efficient compute of scaled_global_grad_norm
#5493

Merged

re-introduce: stage3: efficient compute of scaled_global_grad_norm #5493

lekurile merged 2 commits into deepspeedai:master from nelyahu:offload_fix

Revert "Revert "stage3: efficient compute of scaled_global_grad_norm …

63a89be1

fix for complete_grad_norm_calc in stage3

95aee34f

nelyahu requested a review from

tjruwase 1 year ago

nelyahu requested a review from

mrwyattii 1 year ago

lekurile requested a review from

lekurile 1 year ago

lekurile approved these changes on 2024-05-02

lekurile merged 90793aab into master 1 year ago

nelyahu deleted the offload_fix branch 1 year ago

Reviewers

lekurile

tjruwase

mrwyattii

Assignees

No one assigned

Labels

None yet

Milestone

No milestone