Refactor qconv to reduce allocations. (#42007)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/42007
zero buffer and indirection pointers are allocatoed on every iterations.
With this refactor we create op once for qnnpackconv struct and keep
repopulating indirection pointer as necessary.
For deconv moved much of op creation outside so that we can avoid creating and
destroying ops every time.
Test Plan:
CI quantization tests.
deconvolution-test
Imported from OSS
Reviewed By: AshkanAliabadi
Differential Revision: D22726972
fbshipit-source-id: 07c03a4e90b397c36aae537ef7c0b7d81d4adc1a
