DeepSpeed
d3550dc8 - Adagrad support in ZeRO (#3401)

Commit

2 years ago

Adagrad support in ZeRO (#3401) * Adding torch.optim.Adagrad * adding adagrad for zero 1 2 * Adding Adagrad support to zero 3. * Adding documentation and DeepSpeedCPUAdagrad to list. --------- Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>

References

#3401 - Adagrad support in ZeRO

Author

jomayeri

Parents

77ebf760

Files4

deepspeed/runtime
- engine.py
- zero
  - stage3.py
  - stage_1_and_2.py
  - utils.py

DeepSpeed d3550dc8 - Adagrad support in ZeRO (#3401)

DeepSpeed
d3550dc8 - Adagrad support in ZeRO (#3401)