pytorch
560786ac - call contiguous on BMM inputs for NT on CUDA (#88108)

Commit
2 years ago
call contiguous on BMM inputs for NT on CUDA (#88108) Fixes #87713 BMM for cpu supports non-contiguous nested tensor inputs, while BMM for Cuda does not support currently non-contiguous inputs. The derivative for BMM: ``` - name: bmm(Tensor self, Tensor mat2) -> Tensor self: grad.bmm(mat2.transpose(1, 2).conj()) mat2: self.transpose(1, 2).conj().bmm(grad) result: self_t.bmm(mat2_p) + self_p.bmm(mat2_t) ``` When calling backward it was impossible for this function to succeed since the inputs were always discontiguous, regardless of the user input. This adds contiguous calls to BMM_cuda implementation for nested tensors. This was not caught by tests because grad_check is currently only done on CPU in test_nestedtensors. This PR updates the autograd test to also be run on GPU. As a result I found one more issue with the backward for to_padded_tensor erroring instead of calling the generic version. cc @cpuhrsch @jbschlosser @bhosmer @mikaylagawarecki Pull Request resolved: https://github.com/pytorch/pytorch/pull/88108 Approved by: https://github.com/cpuhrsch
Author
Committer
Parents
Loading