transformers

4f2db330 - Use the Hub rotary kernel for inference only on CUDA (#49439)

Use the Hub rotary kernel for inference only on CUDA (#49439) kernels-community/rotary's apply_rotary_transformers matches the torch rotary in the forward (2e-3 relative, bf16) but not in the backward (0.98 relative error on dq and dk), so use_kernels=True training got wrong gradients below every attention layer.