Add support for linear in mkldnn fusion (#51484)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/51484
This PR moves the linear weights of a frozen model to MKLDNN. When the weights are already in MKLDNN, just computing a single linear by converting the input and output from/to mkldnn provides large speedups. I benchmark'd the results of the top 200 shapes in predictor [here](https://www.internalfb.com/phabricator/paste/view/P171537854) (taken from aten::matmul), as well as verified that it sped up popular models. .
Test Plan: Imported from OSS
Reviewed By: navahgar
Differential Revision: D26696698
Pulled By: eellison
fbshipit-source-id: 53d03b9e6956e11b700ee58214e2266e2aa4106a
