[core] Refactor hub attn kernels (#12475)
* refactor how attention kernels from hub are used.
* up
* refactor according to Dhruv's ideas.
Co-authored-by: Dhruv Nair <dhruv@huggingface.co>
* empty
Co-authored-by: Dhruv Nair <dhruv@huggingface.co>
* empty
Co-authored-by: Dhruv Nair <dhruv@huggingface.co>
* empty
Co-authored-by: dn6 <dhruv@huggingface.co>
* up
---------
Co-authored-by: Dhruv Nair <dhruv@huggingface.co>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
