Remove the remaining fambench_dlrm directory (#1108)
Summary:
Pull Request resolved: https://github.com/pytorch/benchmark/pull/1108
Remove the fambench_dlrm directory with requirements.txt file
Reviewed By: jansel
Differential Revision: D38686849
fbshipit-source-id: 29752ce66ebaea7c30f40a9dcf4e7a7af3bf3bce
