DeepSpeed
PyTorch-compatible backward API
#7665
Merged

Commits
  • rename backward prologue method
    Masahiro Tanaka committed 1 year ago
  • refactor loss scaling
    Masahiro Tanaka committed 1 year ago
  • refactor backward
    Masahiro Tanaka committed 1 year ago
  • fix for bf16 optimizer
    Masahiro Tanaka committed 1 year ago
  • simplify preprocess/postprocess of backward
    Masahiro Tanaka committed 1 year ago
  • fix order of backward postprocess
    Masahiro Tanaka committed 1 year ago
  • enable non-scalar backward only for ZeROOptimizer
    Masahiro Tanaka committed 1 year ago
  • fix zero+fp16 case
    Masahiro Tanaka committed 1 year ago
  • add config to enable allow_user_backward
    Masahiro Tanaka committed 1 year ago
  • fix flag for error handling
    Masahiro Tanaka committed 1 year ago
  • resolve conflict
    Masahiro Tanaka committed 1 year ago
  • add test cases
    Masahiro Tanaka committed 1 year ago
  • Merge branch 'master' into tohtana/backward_non_scalar
    tohtana committed 1 year ago
  • fix format
    Masahiro Tanaka committed 1 year ago
  • return scaled loss from engine's backward
    Masahiro Tanaka committed 1 year ago
  • Merge branch 'master' into tohtana/backward_non_scalar
    tohtana committed 1 year ago
  • remove option to enable user backward
    Masahiro Tanaka committed 1 year ago
  • add hook utility
    Masahiro Tanaka committed 1 year ago
  • fix for z2
    Masahiro Tanaka committed 1 year ago
  • fix scaling
    Masahiro Tanaka committed 1 year ago
  • exclude unused params from counter
    Masahiro Tanaka committed 1 year ago
  • set default flag
    Masahiro Tanaka committed 1 year ago
  • handle non-zero optimizer
    Masahiro Tanaka committed 1 year ago
  • call epilogue in engine's backward
    Masahiro Tanaka committed 1 year ago
  • prevent hooks from being called from nested backward
    Masahiro Tanaka committed 1 year ago
  • run post hook fo rz3
    Masahiro Tanaka committed 1 year ago
  • Merge branch 'master' into tohtana/backward_non_scalar
    tohtana committed 1 year ago
  • added comments
    Masahiro Tanaka committed 1 year ago
  • remove hard-coded tolerances
    Masahiro Tanaka committed 1 year ago
  • add test for multiple engines
    Masahiro Tanaka committed 1 year ago
  • update document
    Masahiro Tanaka committed 1 year ago
  • remove deprecated comment
    Masahiro Tanaka committed 1 year ago
  • simplify utility func to count effective grad nodes
    Masahiro Tanaka committed 1 year ago
  • fix combination with leaf module
    Masahiro Tanaka committed 1 year ago
  • refactor tests
    Masahiro Tanaka committed 1 year ago
  • refactor tests
    Masahiro Tanaka committed 1 year ago
  • fix loss scaling
    Masahiro Tanaka committed 1 year ago
  • Merge branch 'master' into tohtana/backward_non_scalar
    tohtana committed 1 year ago
  • Merge branch 'master' into tohtana/backward_non_scalar
    tohtana committed 1 year ago
Loading