DeepSpeed
Enables ZeRO-3 inference
#1514
Merged

Enables ZeRO-3 inference #1514

jeffra merged 22 commits into master from jeffra/no_optim
jeffra
jeffra add initial support for no optimizer init for Z3
bdb90686
jeffra jeffra changed the title Add support for no optimizer init for Z3 [WIP] Add support for no optimizer init for Z3 4 years ago
jeffra add guards and clean-up
054565b6
jeffra Merge branch 'master' into jeffra/no_optim
d4b7c865
jeffra add docstring to dummy optim
7c2075a9
jeffra Merge branch 'jeffra/no_optim' of github.com:microsoft/DeepSpeed into…
c3f594a5
jeffra jeffra marked this pull request as ready for review 4 years ago
jeffra jeffra requested a review from awan-10 awan-10 4 years ago
jeffra jeffra requested a review from cli99 cli99 4 years ago
jeffra jeffra requested a review from conglongli conglongli 4 years ago
jeffra jeffra requested a review from eltonzheng eltonzheng 4 years ago
jeffra jeffra requested a review from minjiaz minjiaz 4 years ago
jeffra jeffra requested a review from niumanar niumanar 4 years ago
jeffra jeffra requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 4 years ago
jeffra jeffra requested a review from samyam samyam 4 years ago
jeffra jeffra requested a review from ShadenSmith ShadenSmith 4 years ago
jeffra jeffra requested a review from tjruwase tjruwase 4 years ago
jeffra jeffra changed the title [WIP] Add support for no optimizer init for Z3 Allow ZeRO-3 to work without an optimizer 4 years ago
jeffra jeffra changed the title Allow ZeRO-3 to work without an optimizer Enables ZeRO-3 inference 4 years ago
jeffra Merge branch 'master' into jeffra/no_optim
e26bc317
jeffra fix regression introduced in SimpleModel refactor
dfdf0253
jeffra Merge branch 'jeffra/no_optim' of github.com:microsoft/DeepSpeed into…
f091fd8f
jeffra Merge branch 'master' into jeffra/no_optim
901c88a0
jeffra all environ var for test port
28ed3729
jeffra Merge branch 'jeffra/no_optim' of github.com:microsoft/DeepSpeed into…
569747d6
jeffra Merge branch 'master' into jeffra/no_optim
d92c9a8a
jeffra align remote_device/pin_memory in ds config with zero.Init
16f32f6b
jeffra update test to use config
2fe7901f
tjruwase
tjruwase commented on 2021-11-16
tjruwase
tjruwase approved these changes on 2021-11-16
jeffra Merge branch 'master' into jeffra/no_optim
da953d3a
jeffra address comment
498af69d
jeffra Merge branch 'jeffra/no_optim' of github.com:microsoft/DeepSpeed into…
3484b6fe
jeffra fix new lines that broke diff
28988a3d
jeffra Merge branch 'master' into jeffra/no_optim
60f284d9
stas00
jeffra skip param offload test on older torch
a7140c13
jeffra Merge branch 'master' into jeffra/no_optim
346e85e9
jeffra fix import typo
3d21b50f
jeffra jeffra enabled auto-merge (squash) 4 years ago
jeffra jeffra merged 2332cb31 into master 4 years ago
jeffra jeffra deleted the jeffra/no_optim branch 4 years ago

Login to write a write a comment.

Login via GitHub

Assignees
No one assigned
Labels
Milestone