Implement hardswish/hardsigmoid on MKLDNN tensors (#55218)
Summary:
Adding hardwish and hardsigmoid improves mobilenetv3 by ~13%
| hardswish | base |
-- | -- | -- | --
run 1 | 1305.032 | 1486.013 |
run 2 | 1290.142 | 1491.001 |
run 3 | 1305.51 | 1491.66 |
run 4 | 1308.788 | 1495.577 |
avg | 1302.368 | 1491.063 | 0.873449
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55218
Reviewed By: albanD
Differential Revision: D27701276
Pulled By: Krovatkin
fbshipit-source-id: cde78da71d327e65461e80fbb6c3bb3429505410
