Remove duplicate CI jobs between pull and trunk (#94426)
These configs are already in the pull settings and so run on trunk.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/94426
Approved by: https://github.com/malfet, https://github.com/huydhn
