ZeRO-3 partial offload: fix LR schedules and checkpoints of the device sub-groups (#8973)
## Summary
With `offload_optimizer.ratio` < 1, ZeRO-3 keeps some sub-groups on the
device and steps them with a separate torch
optimizer (`backup_optimizer`). That optimizer has several bugs that
change the training result without an error.
SuperOffload uses the same optimizer for its device sub-groups; the rest
of SuperOffload is fixed in #8972.
## Fixes
1. The device sub-groups ignored LR schedules and `set_lr()`, and
changes of `betas`, `eps` and `weight_decay`. Ina
first step at lr 0, the weights moved by 3.0e-3. The backup optimizer
now copies these values before each step.
2. Checkpoints had no Adam state for the device sub-groups (a resume was
4.9e-3 off after one step). It is now saved
with the state of the CPU sub-groups: a resume is bit-identical, and a
checkpoint loads with another `ratio`.
3. After a checkpoint load, every device step also updated the CPU
sub-groups of param groups 1 and up, because their
dicts were shared with the main optimizer (3.0e-3 extra per step at lr
3e-3). Each param group of the backup
optimizer now has its own dict.
4. `safe_get_full_optimizer_state` and `safe_set_full_optimizer_state`
raised `KeyError` for the device sub-groups, and
`ds_to_universal` raised `KeyError: 0`. Both work now.
5. With `adam_w_mode: false`, the device sub-groups used decoupled
weight decay (2.4e-2 off after 4 steps); they now use
`torch.optim.Adam` (L2). `bias_correction: false` with `ratio` < 1 now
raises a `ValueError`.
---------
Signed-off-by: Xinyu Lian <lian7@illinois.edu>