DeepSpeed
Optimize grad_norm calculations by reducing device/host dependency
#4974

Merged

Optimize grad_norm calculations by reducing device/host dependency #4974

mrwyattii merged 6 commits into deepspeedai:master from nelyahu:stage_1_and_2_perf_opt

nelyahu requested a review from

tjruwase 1 year ago

nelyahu requested a review from

mrwyattii 1 year ago

tjruwase approved these changes on 2024-01-18

nelyahu force pushed from 2d828e0d to 7a9229bf 1 year ago

nelyahu force pushed from cedf74f2 to 9b424a25 1 year ago

Optimize grad_norm calculations by reducing device/host dependency

6bd8edd4

nelyahu force pushed from 727e2c60 to 6bd8edd4 1 year ago

Merge branch 'microsoft:master' into stage_1_and_2_perf_opt

94566598

Merge branch 'master' into stage_1_and_2_perf_opt

fbd955cb

Merge branch 'master' into stage_1_and_2_perf_opt

e5077daf

Merge branch 'master' into stage_1_and_2_perf_opt

1dbb7120

Merge branch 'master' into stage_1_and_2_perf_opt

82b6656f

mrwyattii merged 61daaa1e into master 1 year ago

nelyahu deleted the stage_1_and_2_perf_opt branch 1 year ago

Reviewers

tjruwase

mrwyattii

Assignees

No one assigned

Labels

None yet

Milestone

No milestone