transformers
7de47902 - Fix DeepSpeed Trainer tests against deepspeed>=0.19.6

Commit
71 days ago
Fix DeepSpeed Trainer tests against deepspeed>=0.19.6 `deepspeed>=0.19.6` rejects a non-positive `warmup_num_steps` in `WarmupLR`, `WarmupDecayLR` and `WarmupCosineLR`, while older versions silently clamped it to `max(2, warmup_num_steps)`. `trainer_config_finalize` fills the DeepSpeed scheduler config with `args.get_warmup_steps()`, which is `0` for the default `warmup_steps=0`/`warmup_ratio=0.0`, so every DeepSpeed Trainer run died with `ValueError: warmup_num_steps must be a positive integer, got 0`. Keep the auto-filled value positive. `1` is clamped to `2` by DeepSpeed just as `0` used to be, so no-warmup runs behave exactly as before. Also give `test_auto_batch_size_with_deepspeed` a rank environment. It builds a `Trainer` in-process rather than under a launcher, and DeepSpeed refuses to initialize without `LOCAL_RANK`, so it failed with an assertion from `_do_args_sanity_check`. Every other in-process DeepSpeed Trainer test already wraps itself in `mockenv_context(**dist_env_1_gpu)`.
Author
Parents
Loading