transformers
bb3ffb97 - CPU can incur a slow path on non-contiguous magnitudes (#47351)

Commit
123 days ago
CPU can incur a slow path on non-contiguous magnitudes (#47351) * Make Whisper mel magnitudes contiguous to avoid slow strided matmul `stft[..., :-1]` produces a non-contiguous view, and squaring its magnitude keeps it non-contiguous. On some CPU backends (observed on a ROCm PyTorch CPU build) the subsequent `mel_filters.T @ magnitudes` matmul falls onto a pathological strided-GEMM path that runs ~8x slower (~43 ms vs ~3 ms for a 30 s audio window), dominating Whisper feature extraction. Forcing `magnitudes` contiguous costs ~0.18 ms but restores the fast matmul path, cutting per-request audio preprocessing substantially with no change in output. Signed-off-by: Varalakshmi Bayanagari <varalakshmi.bayanagari@amd.com> * Added test case to assert and benchmark * enabled warmup in test case * Remove median timing function from tests Removed median timing measurement for mel_filters matrix multiplication. * Apply make style (ruff) Signed-off-by: vara lakshmi bayanagari <varalakshmi.bayanagari@amd.com> * Reference PR #47351 in test docstring; drop stale -s note Signed-off-by: vara lakshmi bayanagari <varalakshmi.bayanagari@amd.com> --------- Signed-off-by: Varalakshmi Bayanagari <varalakshmi.bayanagari@amd.com> Signed-off-by: vara lakshmi bayanagari <varalakshmi.bayanagari@amd.com>
Author
Parents
Loading