Remove redundant code from LayerNorm Fake Op. (#59054)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/59054
to handle elementwise_affine
Test Plan: GLOW_NNPI=1 buck run -c fbcode.platform=platform009 //caffe2/caffe2/contrib/fakelowp/test:test_layernorm_nnpi_fp16nnpi
Reviewed By: hx89
Differential Revision: D28726804
fbshipit-source-id: b03485e98d490d4e9e1b178a8c50677b77e27596
