Back out "[pytorch][PR] Preserve memory format in qconv op" (#49994)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/49994
Revert preserving memory format in qconv op because it is negatively affecting performance, will revert revert after fixing all issues
Test Plan: pytest fbcode/caffe2/test/quantization/test_quantized_op.py
Reviewed By: kimishpatel
Differential Revision: D25731279
fbshipit-source-id: 908dbb127210a93b27ada7ccdfa531177edf679a
