DeepSpeed
ba58682a - fix errors when setting zero3 leaf modules with torch.compile (#6564)

Commit
1 year ago
fix errors when setting zero3 leaf modules with torch.compile (#6564) When setting zero3 leaf modules to a higher level module and running with torch.compile, there are a few errors from ZeROOrderedDict. First it doesn't support Deep copy for not having a constructor with no parameters. Second, it doesn't check the existence of ds_status attr on param before accessing the attr. change contributed by Haifeng Chen Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>
Parents
Loading