transformers
2126f65b - Fall back to the AITER kernel for flash_attention_2 on ROCm (#49608)

Commit
12 hours ago
Fall back to the AITER kernel for flash_attention_2 on ROCm (#49608) * Fall back to the AITER kernel for flash_attention_2 on ROCm kernels-community/flash-attn2 has no ROCm build, so flash_attention_2 failed on ROCm when the flash-attn package is not installed. Use kernels-community/aiter-flash-attn there, as flash_attention_3 already does. * Add a ROCm test for the flash_attention_2 AITER fallback
Parents
Loading