Remove brew libomp from binary mac machines (#20228)
Summary:
Resolves https://github.com/pytorch/pytorch/issues/20030
Pull Request resolved: https://github.com/pytorch/pytorch/pull/20228
Differential Revision: D15246258
Pulled By: pjh5
fbshipit-source-id: f57033af74b678566be02cdf5700b5ae6e154d4a
