DeepSpeed
BF16 optimizer: Clear lp grads after updating hp grads in hook
#5328

Merged

BF16 optimizer: Clear lp grads after updating hp grads in hook #5328

tjruwase merged 2 commits into deepspeedai:master from YangQun1:yq/fix-lp-grads-clear

BF16 optimizer: Clear lp grads after updating hp grads in hook

6c6c5e53

YangQun1 requested a review from

mrwyattii 1 year ago

YangQun1 requested a review from

tjruwase 1 year ago

Merge branch 'master' into yq/fix-lp-grads-clear

426af61a

tjruwase approved these changes on 2024-04-01

tjruwase merged 40009eb1 into master 1 year ago

Reviewers

tjruwase

mrwyattii

Assignees

No one assigned

Labels

None yet

Milestone

No milestone