Factor out non tensor peephole (#55977)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/55977
Test Plan: Imported from OSS
Reviewed By: nikithamalgifb
Differential Revision: D27755329
Pulled By: eellison
fbshipit-source-id: 0e8948c0607fa59133310e4db8e05ac6847c9f8b
