DeepSpeed
170b46e8 - Add conditional on torch version for scaled_dot_product_attention (#6517)

Commit
1 year ago
Add conditional on torch version for scaled_dot_product_attention (#6517) Changes from #4724 broke support for torch<2.0 in the flops profiler as the scaled_dot_product_attention [wasn't added](https://pytorch.org/docs/2.0/generated/torch.nn.functional.scaled_dot_product_attention.html#torch.nn.functional.scaled_dot_product_attention) until a beta version in torch 2.0 Resolved: #5534 Todo: - [ ] Test this - [ ] Issue resolution with users.
Author
Parents
Loading