Use the correct batch size for timm ViT to improve stableness (#1419)
Summary:
Use the correct batch size for ViT models train.
Upstream batch size: 32 (https://github.com/rwightman/pytorch-image-models/blob/main/train.py#L131)
BS=8, variation 17%: https://github.com/pytorch/benchmark/actions/runs/4147202172
BS=32, variation 0.9%: https://github.com/pytorch/benchmark/actions/runs/4205506091
Pull Request resolved: https://github.com/pytorch/benchmark/pull/1419
Reviewed By: aaronenyeshi, davidberard98
Differential Revision: D43410850
Pulled By: xuzhao9
fbshipit-source-id: c041982ded0dff8aac222e29d8ec9b16f03b161b