Prevent the TE fuser from getting datatypes it can't handle (#44160)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/44160
Test Plan: Imported from OSS
Reviewed By: navahgar
Differential Revision: D23528508
Pulled By: bertmaher
fbshipit-source-id: 03b22725fb2666f441cb504b35397ea6d155bb85
