DeepSpeed
ce0ebdad - [Bug fix] WarmupCosineLR issues (#4688)

Commit
2 years ago
[Bug fix] WarmupCosineLR issues (#4688) Original code missing a `self.` before `warmup_num_steps` so that `warmup_num_steps` might be 0 and cause math domain error when doing `math.log(0)` ```py self.warmup_num_steps = max(2, warmup_num_steps) self.inverse_log_warm_up = 1.0 / math.log(warmup_num_steps) ```
Author
Parents
Loading