PR #4436 optimize grad_norm calculation in stage3.py

optimize grad_norm calculation in stage3.py #4436

tjruwase merged 13 commits into deepspeedai:master from mmhab:optimize_grad_norm_calc

optimize grad_norm calculation

a56838a2

mmhab requested a review from

jeffra 2 years ago

mmhab requested a review from

tjruwase 2 years ago

mmhab requested a review from

samyam 2 years ago

mmhab requested a review from

mrwyattii 2 years ago

tjruwase commented on 2023-10-03

Merge branch 'master' into optimize_grad_norm_calc

671de1cd

Merge branch 'master' into optimize_grad_norm_calc

ecfda40d

Merge branch 'master' into optimize_grad_norm_calc

62d52aa8

tjruwase approved these changes on 2023-12-13

Merge branch 'master' into optimize_grad_norm_calc

d78ee555

Merge branch 'master' into optimize_grad_norm_calc

6ac9f7d0

Merge branch 'master' into optimize_grad_norm_calc

f180a44d

mrwyattii commented on 2023-12-18

tjruwase commented on 2023-12-18

Update deepspeed/runtime/zero/stage3.py

93461bb7

Update deepspeed/runtime/zero/stage3.py

038d7555

Merge branch 'master' into optimize_grad_norm_calc

8cca05b8

Merge branch 'master' into optimize_grad_norm_calc

799e9a24

optimize grad_norm calculation

7ef1be9a

Merge branch 'master' into optimize_grad_norm_calc

753b3533

tjruwase merged ea0d8114 into master 1 year ago

Reviewers

tjruwase

mrwyattii

jeffra

samyam

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed optimize grad_norm calculation in stage3.py #4436 Merged

optimize grad_norm calculation in stage3.py #4436

DeepSpeed
optimize grad_norm calculation in stage3.py
#4436

Merged