[FX][EZ] Hoist custom class .so loading into setUp (#52883)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/52883
Test Plan: Imported from OSS
Reviewed By: suo
Differential Revision: D26675802
Pulled By: jamesr66a
fbshipit-source-id: 7a7bcb1d0a6f8c9b1431bc3e09143ada6e5fbf4d
