DeepSpeed
c66fdaf3 - Make sure it's not None before offloading contiguous_grad_buffer (#7227)

Comment changes are shownComment changes are hidden
Commit
68 days ago
Make sure it's not None before offloading contiguous_grad_buffer (#7227) Resolves #7223 When DeepCompile is enabled in ZeRO-3, contiguous_grad_buffer is released, so we should check and make sure it's not None before we continue. https://github.com/deepspeedai/DeepSpeed/blob/227a60c0c412ddf4619401b5d8d9d1674aee17b5/deepspeed/compile/init_z3.py#L22-L25 Signed-off-by: Hollow Man <hollowman@opensuse.org> Co-authored-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com>
Author
Parents
  • deepspeed/runtime/zero
    • File
      stage3.py