BatchNorm: error on a single value per channel in training (fixes #1992) (#2716)
* BatchNorm: error on single value per channel in training (#1992)
A channel that sees exactly one value in training (e.g. batch size 1 with
no spatial dimensions) has an undefined batch variance, and the
Bessel-corrected running-variance update `m/(m-1)·σ²` becomes `0/0 = NaN`,
silently corrupting the tracked statistics. cuDNN emits NaN here too; PyTorch
rejects the batch outright.
Guard the `BatchNorm` layer forward pass so it throws an informative
`ArgumentError` when training and there is a single value per channel,
matching PyTorch. The check sits before dispatch, so CPU and GPU behave
consistently. `testmode!` inference with batch size 1 is unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix typo in v0.16.11 release date
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Apply suggestion from @CarloLucibello
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>