Revert "fix inference_mode with torch.compile (#101219)"
This reverts commit 11f7ae19cd068e7b01b7a296db869b4933951a57.
Reverted https://github.com/pytorch/pytorch/pull/101219 on behalf of https://github.com/PaliC due to breaking inductor tests ([comment](https://github.com/pytorch/pytorch/pull/101219#issuecomment-1555104220))
