Revert "Run all of the timm models shards in the periodic (#92743)"
This reverts commit de69cedf98ae578f26add662c6387a43cf098066.
Reverted https://github.com/pytorch/pytorch/pull/92743 on behalf of https://github.com/atalman due to This needs to be landed after https://github.com/pytorch/pytorch/pull/92845 and https://github.com/pytorch/pytorch/pull/92846 are landed
