Add defence for offload_states and reload_states w/o optimizer (#7211)
When the optimizer is not specified, the optimizer will be type
`DeepSpeedZeRoOffload` instead of `DeepSpeedZeroOptimizer_Stage3` (e.g.
for ZeRO-3 pure inference), while `DeepSpeedZeRoOffload` hasn't
implemented methods `reload_states` and `offload_states`.
https://github.com/deepspeedai/DeepSpeed/blob/56005d2b256eb81a88cba0a1984375f9663a3110/deepspeed/runtime/engine.py#L1684-L1707
```log
File "deepspeed/runtime/engine.py", line 3904, in offload_states
self.optimizer.offload_states(include=include, device=device, pin_memory=pin_memory, non_blocking=non_blocking)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'DeepSpeedZeRoOffload' object has no attribute 'offload_states'
```
In addition, https://github.com/deepspeedai/DeepSpeed/pull/6855 seems to
forget removing the check for `assert not self.zero_offload_param()`, as
suggested by
https://github.com/deepspeedai/DeepSpeed/issues/6833#issuecomment-2537295310,
it returns None when offload_param is not given, and the newly added
assertions have already covered these cases. This PR also removed this
old check.
Signed-off-by: Hollow Man <hollowman@opensuse.org>
DeepSpeed
a21e5b9d - Add defence for offload_states and reload_states w/o optimizer (#7211)
