Update on "Complex gradcheck logic"
This PR adds gradcheck for complex. The logic used for complex gradcheck is described in Section 3.5.3 here: https://arxiv.org/pdf/1701.00392.pdf
More concretely, this PR introduces the following changes:
1. Updates get_numerical_jacobian to take as input a scalar value for vector (v). Adds gradcheck logic for C -> C, C-> R, R -> C. For R -> C functions, only the real value of gradient is propagated.
2. Adds backward definition for `torch.complex` and also adds a test to verify the definition added.
3. Updates backward for `mul`, `sin`, `cos`, `sinh`, `cosh`.
4. Adds tests for all `torch.real`, `torch.imag`, `torch.view_as_real`, `torch.view_as_complex`, `torch.conj`.
Follow up tasks:
- [ ] Add more thorough tests for R -> C cases. Specifically, add R->C test variants for functions. for e.g., `torch.mul(complex_tensor, real_tensor)` (#44744 )
- [ ] Add back commented test in `common_methods_invocation.py`. ( #43208 )
- [ ] Add more special case checking for complex gradcheck to make debugging easier.
- [ ] Update complex autograd note.
- [ ] disable complex autograd for operators not tested for complex.
- [ ] Re-enable tests in `test_ops.py` for complex dtype. ( #43208 )
- [ ] Re-enable `TestGradCheckOverride.test_gradcheck` cc. @hameerabbasi
Differential Revision: [D23655088](https://our.internmc.facebook.com/intern/diff/D23655088)
[ghstack-poisoned]
