DeepSpeed
Support fp32 gradaccum for bf16 model
#2566
Merged
Overview
Commits
5
Changes
View On GitHub
Commits
  • allow bf16 model with fp32 gradient accumulation datatype
    delock committed 3 years ago
  • allow fp32 gradient accumulation and bfloat16 model in amp mode
    delock committed 3 years ago
  • Merge branch 'master' into gma/support_fp32_gradaccum_for_bf16_model
    tjruwase committed 3 years ago
  • alternative fix for grad accumulation type mismatch. In the case of zero optimizer we should have grad accum type == model data type
    delock committed 3 years ago
  • Merge branch 'master' into gma/support_fp32_gradaccum_for_bf16_model
    tjruwase committed 3 years ago
FAQ Terms Privacy Refunds Impressum
Loading