Keep invalid group-norm sentinels from becoming a finite 1.0 step (#8638)
## Summary
- DeepSpeed encodes inf/NaN group norms as `-1`. Combining those with
`vector_norm` squares the sentinel to `1.0`, so clipping and Adam treat
a failed group as healthy and apply an unclipped step.
- Treat non-finite or negative group norms as overflow (`inf`), skip the
optimizer step, reset overflow each step, and advance the loss scaler
once (including ZeRO-3 / SuperOffload / ZenFlow / FP16 fused+unfused /
BF16).
- Direct `unscale_and_clip_grads` callers map the sentinel to `inf` so
they still zero gradients instead of using clip_coef=1.
This is split out of the ZeRO-2 oversized-gradient / stream-lifetime
work so the sentinel contract can land independently.
## Test plan
- [x] CPU, rebased onto current master (`75eb961d1`, includes #8730): 17
passed in `tests/unit/v1/zero/test_invalid_grad_norm.py`
(`combine_grad_norm_groups` / `get_global_norm` / `CheckOverflow`,
ZeRO-1 clip, CPU-offload combine, ZeRO-3 / BF16 / SuperOffload overflow
paths)
- [x] GPU (2× H200, prior run):
`TestZeROBFloat16Stage3InvalidNormRecovery`. Skipped on the rebase
machine (2 GPUs required, 1 available)
- [ ] CI unit tests for the touched runtime files, started by the rebase
push
---------
Signed-off-by: tunji-ruwase_snow <tunji.ruwase@snowflake.com>
Co-authored-by: Cursor <cursoragent@cursor.com>