DeepSpeed
f5aa2547 - Add CPUAdam optimizer for zero-offload in deepspeed engine (#484)
Commit
View On GitHub
Commit
4 years ago
Add CPUAdam optimizer for zero-offload in deepspeed engine (#484) * add adamW to CPU-ADAM implementation * supporting cpu-adam optimizer for zero-offload on deepspeed side * bump DSE to match cpu-adam updates Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Parents
d720fdb6
  • File
    DeepSpeedExamples
  • csrc
    • adam
      • File
        cpu_adam.cpp
    • includes
      • File
        cpu_adam.h
  • deepspeed
    • File
      __init__.py
    • ops/adam
      • File
        cpu_adam.py
    • runtime
      • File
        config.py
      • File
        engine.py
      • zero
        • File
          stage2.py
  • docs
    • _pages
      • File
        config-json.md
    • code-docs/source
      • File
        optimizers.rst
  • tests/unit
    • File
      test_cpu_adam.py
FAQ Terms Privacy Refunds Impressum