Unify code path for native mkldnn conv and reorder on-the-fly mkldnn conv (#19963)
Summary:
Also, current mkldnn primitive code recreate the computation everytime, causes tiny Convolutions spending significant portion of its time on the repeated codegen. ideep has implemented an lru cache to save the computation and so this change will help us improving perfs for tiny Convolutions.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/19963
Differential Revision: D15156527
Pulled By: bddppq
fbshipit-source-id: 6a8fbd10a213ec22cdeaff1a2bdb0d09905d1fcd
