Flux.jl
3f2f828b - BatchNorm: error on a single value per channel in training (fixes #1992) (#2716)

Commit
62 days ago
BatchNorm: error on a single value per channel in training (fixes #1992) (#2716) * BatchNorm: error on single value per channel in training (#1992) A channel that sees exactly one value in training (e.g. batch size 1 with no spatial dimensions) has an undefined batch variance, and the Bessel-corrected running-variance update `m/(m-1)·σ²` becomes `0/0 = NaN`, silently corrupting the tracked statistics. cuDNN emits NaN here too; PyTorch rejects the batch outright. Guard the `BatchNorm` layer forward pass so it throws an informative `ArgumentError` when training and there is a single value per channel, matching PyTorch. The check sits before dispatch, so CPU and GPU behave consistently. `testmode!` inference with batch size 1 is unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Fix typo in v0.16.11 release date Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Apply suggestion from @CarloLucibello --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Parents
Loading