Revert "moved logit to use torch ops instead of refs + added a couple more decompositions"
This reverts commit 1d9f445b5d130846fd69512b84a86e6b3fc78b64.
Reverted https://github.com/pytorch/pytorch/pull/78984 on behalf of https://github.com/osalpekar due to broke some jobs, like meta functorch builds
