Add subchannel FP8/INT8 quantized training for nn.Linear and AutoEP experts
A new opt-in `quantization.subchannel` config section quantizes the three
GEMMs of a linear layer (forward, dx, dw) independently: FP8 (e4m3/e5m2) or
INT8 operands with one FP32 scale per block of the reduction dimension,
optional stochastic rounding, and an optional fused butterfly Hadamard
rotation inside each block. `block_size: 0` gives each operand row a single
scale, which lets the FP8 GEMM apply scales once per output element and run
faster than BF16 on wide layers.
The GEMMs run on Triton kernels on CUDA devices of compute capability 8.9+
(8.0+ for INT8) and on a PyTorch reference implementation elsewhere.
Parameters, gradients and optimizer state keep their usual precision.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: Zhipeng Wang <7927074+PKUWZP@users.noreply.github.com>