DeepSpeed
ZeRO-1/2: fix offload_states() keeping the optimizer state on the device
#9016
Merged

Commits
  • ZeRO-1/2: free the optimizer state in offload_states() and relink it on reload
    xylian86 committed 5 hours ago
  • Potential fix for pull request finding 'Defer elastic state linking until optimizer state is initialized'
    xylian86 committed 5 hours ago
  • ZeRO-1/2: test offload and reload before the first step
    xylian86 committed 5 hours ago
  • Merge branch 'master' into xinyu/zero12-offload-states-optim-fragments
    xylian86 committed 5 hours ago
Loading