Revert "Ignore shape inference exception from Caffe2 ATen fallback (#90408)"
This reverts commit 1d4e8723705280a82497d366cdf37e6aef49725d.
Reverted https://github.com/pytorch/pytorch/pull/90408 on behalf of https://github.com/huydhn due to Sorry for reverting your PR, but it breaks lint check https://hud.pytorch.org/pr/90408#11855039599. Please fix the error and reland your change
