DeepSpeed
Enable torch.compile with ZeRO (Experimental)
#4878
Merged
Overview
Commits
49
Changes
View On GitHub
Commits
  • add option to run torch.compile
    Masahiro Tanaka committed 2 years ago
  • improve compile helper
    Masahiro Tanaka committed 2 years ago
  • fix compile wrapper to make modules() work
    Masahiro Tanaka committed 2 years ago
  • add torch.compiler-disable to comm module
    Masahiro Tanaka committed 2 years ago
  • move options for torch.compile to ds config
    Masahiro Tanaka committed 2 years ago
  • Merge branch 'master' into tohtana/compile-zero
    Masahiro Tanaka committed 2 years ago
  • rename module and wrap decorator
    Masahiro Tanaka committed 2 years ago
  • fix validation of compile config
    Masahiro Tanaka committed 2 years ago
  • avoid reference to torch._dynamo when torch has no support
    Masahiro Tanaka committed 2 years ago
  • fix custom backend for test
    Masahiro Tanaka committed 2 years ago
  • fix validation
    Masahiro Tanaka committed 2 years ago
  • refactor config for torch.compile
    Masahiro Tanaka committed 2 years ago
  • Merge branch 'master' into tohtana/compile-zero
    tohtana committed 2 years ago
  • Merge branch 'master' into tohtana/compile-zero
    tohtana committed 2 years ago
  • fix validation of compiler config
    Masahiro Tanaka committed 2 years ago
  • fix access to wrapped model
    Masahiro Tanaka committed 2 years ago
  • add test for api to set torch compile options
    Masahiro Tanaka committed 2 years ago
  • rename util module
    Masahiro Tanaka committed 2 years ago
  • fix import
    Masahiro Tanaka committed 2 years ago
  • Merge branch 'master' into tohtana/compile-zero
    tohtana committed 2 years ago
  • delay reduce-scatter for z3 leaf modules
    Masahiro Tanaka committed 2 years ago
  • Merge branch 'master' into tohtana/z3_moe_bwd
    tohtana committed 2 years ago
  • add comment to config class
    Masahiro Tanaka committed 2 years ago
  • Merge branch 'master' into tohtana/compile-zero
    tohtana committed 2 years ago
  • add api to get leaf modules
    Masahiro Tanaka committed 2 years ago
  • Merge branch 'master' into tohtana/compile-zero
    tjruwase committed 2 years ago
  • Merge branch 'master' into tohtana/z3_moe_bwd
    tjruwase committed 2 years ago
  • add api to set a function to run torch.compile
    Masahiro Tanaka committed 2 years ago
  • Merge branch 'master' into tohtana/compile-zero
    mrwyattii committed 2 years ago
  • refactor compile config
    Masahiro Tanaka committed 2 years ago
  • lift is_compile_supported up to use as `deepspeed.is_compile_supported()`
    Masahiro Tanaka committed 2 years ago
  • avoid overwriting backend fn in validator
    Masahiro Tanaka committed 2 years ago
  • add tests combining compile and zero
    Masahiro Tanaka committed 2 years ago
  • rename test modules
    Masahiro Tanaka committed 2 years ago
  • Merge branch 'master' into tohtana/z3_moe_bwd
    tohtana committed 2 years ago
  • Merge branch 'master' into tohtana/z3_moe_bwd
    tohtana committed 2 years ago
  • use no zero + no compile as baseline for tests
    Masahiro Tanaka committed 2 years ago
  • disable memory_efficient_linear when torch.compile is enabled
    Masahiro Tanaka committed 2 years ago
  • pass only tensors to z3 hooks to prevent dynamo from displaying errors
    Masahiro Tanaka committed 2 years ago
  • Merge branch 'tohtana/z3_moe_bwd' into tohtana/compile-zero
    Masahiro Tanaka committed 2 years ago
  • Merge branch 'master' into tohtana/compile-zero
    Masahiro Tanaka committed 2 years ago
  • fix exception used in test
    Masahiro Tanaka committed 2 years ago
  • increse tolerance in tests
    Masahiro Tanaka committed 2 years ago
  • add check for bf16
    Masahiro Tanaka committed 2 years ago
  • enable accelerator check for bf16
    Masahiro Tanaka committed 2 years ago
  • Merge branch 'master' into tohtana/compile-zero
    tohtana committed 2 years ago
  • update DistributedTest to work with torch.compile tests
    mrwyattii committed 2 years ago
  • remove unused global
    mrwyattii committed 2 years ago
  • Merge branch 'master' into tohtana/compile-zero
    tohtana committed 2 years ago
FAQ Terms Privacy Refunds Impressum
Loading