DeepGEMM (#44832)
* deep gemm
* standardize
* clear deepgemm flow and blackwell optimization
* avoid unnecessary gathers in grouped mm
* assertions and drop the synced path
* use lazy load kernel
* style
* add prefix and check for cuda runtime version
* better names
* exit on missing functions
* comment about why we use deepspeed cutlass single gemm
* global statements
* add cuda check
* fix deepgemm fastpath for models with bf16 scales
* force fp32 scales in experts as well
* Update src/transformers/integrations/finegrained_fp8.py
Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com>
* Update src/transformers/integrations/hub_kernels.py
Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com>
* fix
* style
---------
Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com>
