[Quant] qconv: fix xnnpack operator caching (#74217)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74217
This makes the caching work as intended.
Test Plan: CI
Reviewed By: kimishpatel
Differential Revision: D34879710
fbshipit-source-id: 76d2e884a88dd619187007c2974ed935db356286
(cherry picked from commit f0a5726e8ab616f51c858530038a0f35c6f77813)
