DeepSpeed
Fix allreduce for BF16 and ZeRO0
#5170
Merged
Overview
Commits
5
Changes
View On GitHub
Commits
  • fix gradient clipping
    Masahiro Tanaka committed 1 year ago
  • Merge branch 'master' into tohtana/fix_fp32_clipping
    loadams committed 1 year ago
  • perform allreduce on FP32 when BF16 optimizer is enabled
    Masahiro Tanaka committed 1 year ago
  • ZZMerge branch 'master' into tohtana/fix_bf16_z0_reduce
    Masahiro Tanaka committed 1 year ago
  • Merge branch 'master' into tohtana/fix_bf16_z0_reduce
    tohtana committed 1 year ago
FAQ Terms Privacy Refunds Impressum
Loading