transformers
2649c0d4 - Keep AFMoE expert biases in FP32 on BF16 load (#50687)

Commit
1 day ago
Keep AFMoE expert biases in FP32 on BF16 load (#50687) * Preserve AFMoE normalization weights and expert biases on BF16 load Signed-off-by: Anneketh Vij <142443606+annekethvij@users.noreply.github.com> * Keep only AFMoE expert biases in FP32 on BF16 load Signed-off-by: Anneketh Vij <142443606+annekethvij@users.noreply.github.com> --------- Signed-off-by: Anneketh Vij <142443606+annekethvij@users.noreply.github.com>
Author
Parents
Loading