Fix qconv benchmark (#24019)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/24019
Permutes are done inside the module. We don't need them outside.
Setting of scale/zero_point has changed.
Reviewed By: jianyuh
Differential Revision: D16712437
fbshipit-source-id: e3cedf9d63347fbf8070d1a65a196e6d4b2833fc
