Revert "Enable fake tensor caching in fbcode by default (#118555)"
This reverts commit 0f2fbbff109cbc184a6a88247813dbcddaea2e5f.
Reverted https://github.com/pytorch/pytorch/pull/118555 on behalf of https://github.com/huydhn due to Sorry for reverting your change but it is failing one model test internally. Please take a look at the diff for more info D53189048 ([comment](https://github.com/pytorch/pytorch/pull/118555#issuecomment-1939550273))
