Add a few InferenceMode test cases to the wall. (#55993)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/55993
Test Plan: Imported from OSS
Reviewed By: robieta
Differential Revision: D27769478
Pulled By: ailzhang
fbshipit-source-id: 009592d64ef24e1cf7e977d02acf662eb841ca58
