DeepSpeed
Fixes for training models with bf16 + freshly initialized optimizer via `load_module_only`
#4141
Merged
Overview
Commits
8
Changes
View On GitHub

Fixes for training models with bf16 + freshly initialized optimizer via `load_module_only` #4141

loadams merged 8 commits into deepspeedai:master from haileyschoelkopf:new-fix
haileyschoelkopf
haileyschoelkopf commit 3 hacks
1590973f
haileyschoelkopf cleanup
32fc3e92
no longer need [None] check with better fixes
8cbdbaa5
haileyschoelkopf haileyschoelkopf requested a review from jeffra jeffra 2 years ago
haileyschoelkopf haileyschoelkopf requested a review from tjruwase tjruwase 2 years ago
janelu9
Quentin-Anthony
tjruwase
tjruwase
tjruwase approved these changes on 2023-11-07
haileyschoelkopf Merge branch 'master' into new-fix
51aa22bc
haileyschoelkopf
tjruwase Merge branch 'master' into new-fix
37439295
exnx
tjruwase Merge branch 'master' into new-fix
64c77873
haileyschoelkopf haileyschoelkopf requested a review from mrwyattii mrwyattii 1 year ago
loadams Merge branch 'master' into new-fix
e11975fe
loadams Merge branch 'master' into new-fix
5797d3e9
loadams loadams enabled auto-merge 1 year ago
loadams loadams merged 870ae041 into master 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum