DeepSpeed
6e3bd087 - Reject invalid ZenFlow ratio and update interval boundaries (#8274)

Commit
3 days ago
Reject invalid ZenFlow ratio and update interval boundaries (#8274) ## Summary - require `zenflow.topk_ratio` to be strictly between 0 and 1 - require numeric `zenflow.update_interval` values to be at least 1 - add regression coverage for the invalid ratio endpoints and non-positive update intervals ## Why ZenFlow's automatic update path normalizes selected gradients by `topk_ratio` and unselected gradients by `1 - topk_ratio`. The explicit update path selects its overlap buffer using `micro_step // update_interval`. The previous validation accepted ratio endpoints as well as zero and negative numeric update intervals. These values could therefore trigger division by zero or invalid scheduling after training had already initialized. Rejecting them during configuration parsing provides an immediate validation error while preserving all usable configurations. ## Testing - `python -m pytest -q tests/unit/runtime/zero/test_zero_config.py tests/unit/runtime/zenflow/test_zf_config.py tests/unit/runtime/zenflow/test_zf.py::test_split_affinity` — 21 passed - `pre-commit run --files deepspeed/runtime/zenflow/zenflow_config.py tests/unit/runtime/zenflow/test_zf_config.py` --------- Signed-off-by: tandede <1090179959@qq.com> Co-authored-by: Tingfeng Lan <erc8gx@virginia.edu>
Author
Parents
Loading