Use addmm directly for 1x1 convolution (#45557)
Summary:
Fixes https://github.com/pytorch/pytorch/issues/45274
Based on https://github.com/pytorch/pytorch/issues/44041, sets intermediate for backward computation (otherwise, backward tests are failing).
Pull Request resolved: https://github.com/pytorch/pytorch/pull/45557
Reviewed By: izdeby
Differential Revision: D24030655
Pulled By: ngimel
fbshipit-source-id: 368fe9440668dffc004879f8b1d2dd3787d915c9
