Remove hacky python dispatcher fallthrough (#96635)
Ed's previous PRs in stack https://github.com/pytorch/pytorch/pull/96306 fixes #89037, but this PR just removes the original hacky fallthrough.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/96635
Approved by: https://github.com/zhxchen17
