PR #1487 allreduce_always_fp16

fp16 allreduce

Dipet committed 4 years ago

Merge branch 'master' into allreduce_fp16

jeffra committed 4 years ago

Merge branch 'master' into allreduce_fp16

jeffra committed 4 years ago

Merge branch 'master' into allreduce_fp16

tjruwase committed 4 years ago

Undo sparse sum in nan check

Dipet committed 4 years ago

Merge branch 'master' into allreduce_fp16

tjruwase committed 4 years ago

Merge branch 'master' into allreduce_fp16

Dipet committed 4 years ago

communication_data_type instead of fp32_allreduce and fp16_allreduce

Dipet committed 4 years ago

sparse_allreduce with fp32 or fp16 data type

Dipet committed 4 years ago

Merge branch 'master' into allreduce_fp16

tjruwase committed 4 years ago

FIx communication_data_type checks

Dipet committed 4 years ago

Merge branch 'allreduce_fp16' of github.com:Dipet/DeepSpeed into allreduce_fp16

Dipet committed 4 years ago

Merge branch 'master' into allreduce_fp16

tjruwase committed 4 years ago

Merge branch 'master' into allreduce_fp16

tjruwase committed 4 years ago

Merge branch 'master' into allreduce_fp16

tjruwase committed 4 years ago

Merge branch 'master' into allreduce_fp16

tjruwase committed 4 years ago

Merge Master

Dipet committed 4 years ago

Allow only torch data types for communication_data_type

Dipet committed 4 years ago

Merge branch 'master' into allreduce_fp16

Dipet committed 4 years ago

Fix Zero assert messages

Dipet committed 4 years ago

Merge branch 'master' into allreduce_fp16

tjruwase committed 4 years ago

Merge branch 'master' into allreduce_fp16

tjruwase committed 4 years ago

Merge branch 'master' into allreduce_fp16

tjruwase committed 4 years ago

Merge branch 'master' into allreduce_fp16

tjruwase committed 4 years ago

Merge branch 'master' into allreduce_fp16

Dipet committed 4 years ago

DeepSpeed allreduce_always_fp16 #1487 Merged

DeepSpeed
allreduce_always_fp16
#1487

Merged