Remove unnecessary torchrec-nightly dep on dlrm (#1813)
Summary:
Signed-off-by: Edward Z. Yang <ezyang@meta.com>
Pull Request resolved: https://github.com/pytorch/benchmark/pull/1813
Reviewed By: xuzhao9
Differential Revision: D48167029
Pulled By: ezyang
fbshipit-source-id: 189cdeed8fef08a073cd3f1f55a5cb539ba98451
