Revert "Fix periodic job by excluding check_graph_breaks (#96780)"
This reverts commit 8ec9beacec54fb6c56102f60180063f0fca7f24c.
Reverted https://github.com/pytorch/pytorch/pull/96780 on behalf of https://github.com/wconstab due to broke trunk builds that didn't run on PR? didn't see those trunk failures on PR CI even with trunk label
