Revert D22418731: [JIT] Add out-of-source-tree to_backend tests
Test Plan: revert-hammer
Differential Revision:
D22418731 (https://github.com/pytorch/pytorch/commit/e2a291b396be8db6f9ac1fc7d45e97c6026df590)
Original commit changeset: 621ba4efc1b1
fbshipit-source-id: 475ae24c5b612fe285035e5ebb92ffc66780a468
