PR #7184 Reland perf fix for nan inf check

Revert "Fix issue #5242 grad_norm and loss is nan (#7171)"

Nadav Elyahu committed 1 year ago

replace boolen compute with torch.where for nan/inf detection

Nadav Elyahu committed 1 year ago

move logic to runtime.utils and reuse code

Nadav Elyahu committed 1 year ago

fix formatting issues

Nadav Elyahu committed 1 year ago

Merge branch 'master' into reland_perf_fix_for_nan_inf_check

loadams committed 1 year ago

Merge branch 'master' into reland_perf_fix_for_nan_inf_check

hwchen2017 committed 1 year ago

DeepSpeed Reland perf fix for nan inf check #7184 Merged