Enable V2 workflow (#671)
Summary:
This PR enables V2 nightly workflow on CI.
How we determine test stability:
1. Run the tests on PyTorch 20211222 for 20 times
2. For each test, check if its max delta `(max - min) / min` is above a threshold (we use 7% as threshold)
3. If yes, we determine the test is unstable
In total, we found 210 stable tests and 74 unstable tests, where there are 71 unstable cpu tests, and 3 unstable cuda tests.
Example of the V2 workflow: https://github.com/pytorch/benchmark/actions/runs/1655586619
It takes ~5h to finish the entire suite.
Pull Request resolved: https://github.com/pytorch/benchmark/pull/671
Reviewed By: erichan1
Differential Revision: D33460408
Pulled By: xuzhao9
fbshipit-source-id: 9b9010dfb3cb5335b3313cc6ca5cb78deb4642b2
