DeepSpeed
ceef30ff - fix: attention_unfused alpha missing norm_factor squaring

Commit
31 days ago
fix: attention_unfused alpha missing norm_factor squaring ds_attention passes 1/sqrt(sqrt(head_dim)); attention_unfused used it directly as QK^T scale (head_dim^(-1/4)). The correct softmax scale is head_dim^(-1/2), requiring alpha = norm_factor * norm_factor. The fallback binding in the same file (line 378) already does this — this fix aligns attention_unfused with it. Signed-off-by: Guokai Ma <guokai.ma@intel.com>
Author
Parents
Loading