DeepSpeed
f6a386a3 - Preserve tensor learning rates in OneCycle (#8205)

Commit
3 days ago
Preserve tensor learning rates in OneCycle (#8205) Follow up #8202: When an optimizer starts with a tensor learning rate, OneCycle initialization replaces it with a Python scalar while applying `cycle_min_lr`. This loses the caller's tensor identity, shape, and dtype before later scheduler updates can preserve them. This PR initializes OneCycle learning rates through the existing tensor-aware update helper, matching the path used by subsequent scheduler steps. Signed-off-by: Masahiro Tanaka <mtanaka@anyscale.com>
Author
Parents
Loading