[ROCM] Navi21 Enablement 5: Softmax kernels (#73545)
Summary:
This PR is a follow up to the following prs.
https://github.com/pytorch/pytorch/pull/69942
https://github.com/pytorch/pytorch/pull/72682
https://github.com/pytorch/pytorch/pull/72809
https://github.com/pytorch/pytorch/pull/73543
We are adding support to Navi21 GPUs which have a warpsize of 32. We cannot rely on a constant so we have to dynamically look up the warpsize when launching the kernel on the host side. Inside device functions this is not needed and the compiler can correctly detect the correct warpsize to replace the C10_WARP_SIZE constant.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73545
Reviewed By: jbschlosser
Differential Revision: D34616171
Pulled By: ngimel
fbshipit-source-id: d9b3a17de2457e33ddc5d9c817799a1c85826ccb
(cherry picked from commit f54b12c642e4b33cf9bf27f93f628e147dc37ddc)
