remove SkipInfo class (#64972)
Summary:
per title
Pull Request resolved: https://github.com/pytorch/pytorch/pull/64972
Reviewed By: mruberry
Differential Revision: D30924598
Pulled By: ngimel
fbshipit-source-id: 1ac1ec8fd50ca27e3cd36c12a588d334e7466899
