Fix a bug in hf_bert distributed (#1477)
Summary:
The distributed CI was broken due to a bug from hf_bert e2e model:
https://github.com/pytorch/benchmark/actions/runs/4379172888
Pull Request resolved: https://github.com/pytorch/benchmark/pull/1477
Reviewed By: janeyx99
Differential Revision: D43982886
Pulled By: xuzhao9
fbshipit-source-id: 384e2eb78749a0198d8fbbf28f96002746fd0dab
