DeepSpeed
Fix ZeRO parameter initialization for tensors with `requires_grad=True`
#4138

Merged

Fix ZeRO parameter initialization for tensors with `requires_grad=True` #4138

loadams merged 5 commits into deepspeedai:master from XuehaiPan:detach-dtype-cast

Fix ZeRO parameter initialization for tensors with `requires_grad=True`

05d97f9e

XuehaiPan requested a review from

jeffra 2 years ago

XuehaiPan requested a review from

tjruwase 2 years ago

XuehaiPan requested a review from

samyam 2 years ago

XuehaiPan requested a review from

mrwyattii 2 years ago

tjruwase requested a review from

tohtana 2 years ago

tjruwase removed review request from

jeffra 2 years ago

tjruwase removed review request from

samyam 2 years ago

tjruwase removed review request from

mrwyattii 2 years ago

Simplify detach logic

85ce73bc

Merge branch 'master' into detach-dtype-cast

f67410d5

tohtana approved these changes on 2023-08-17

Merge branch 'master' into detach-dtype-cast

430ae909

loadams enabled auto-merge 2 years ago

Merge branch 'master' into detach-dtype-cast

e27ca495

loadams merged 426810a2 into master 2 years ago

XuehaiPan deleted the detach-dtype-cast branch 2 years ago

Reviewers

tohtana

tjruwase

Assignees

No one assigned

Labels

None yet

Milestone

No milestone