Fix a100 bisection workflow (#2055)
Summary: Pull Request resolved: https://github.com/pytorch/benchmark/pull/2055
Test Plan:
Test workflow: https://github.com/pytorch/benchmark/actions/runs/6985353191
Automatic bisection of the `test_bench` userbenchmark on the accuracy issue of llama_v2_7b_16h:
Start commit hash (2023-11-15): `a5a404865c01f86881f6b3ab0cd9a562d0b420de`
End commit hash (2023-11-16): `690c805c8b539501aad5fbf18914ac92afb65d5a`
Userbenchmark name: `test_bench`
Userbenchmark arguments: `llama_v2_7b_16h -d cuda -t eval --accuracy`
Known root cause commit: `12b2dd16b050e6495910fc564517fbb51dde1f20`
Result:
```
{
"target_repo": "pytorch",
"start": "6f4409073f9fad32016f18b3b083f959d7600a2b",
"end": "a3b859fc673515f74d103494e61b432c9cc9009f",
"result": [
{
"commit1": "cc11c0d11bf",
"commit1_time": "2023-11-15 19:55:47 +0000",
"commit1_digest": {
"name": "test_bench",
"environ": {
"pytorch_git_version": "cc11c0d11bfdcc1f1804cedfcd634a5e79715638",
"pytorch_version": "2.2.0a0+gitcc11c0d",
"device": "NVIDIA A100-SXM4-40GB",
"git_commit_hash": "cc11c0d11bfdcc1f1804cedfcd634a5e79715638"
},
"metrics": {
"model=llama_v2_7b_16h, test=eval, device=cuda, bs=None, extra_args=['--accuracy'], metric=accuracy": "pass"
}
},
"commit2": "12b2dd16b05",
"commit2_time": "2023-11-15 20:26:13 +0000",
"commit2_digest": {
"name": "test_bench",
"environ": {
"pytorch_git_version": "12b2dd16b050e6495910fc564517fbb51dde1f20",
"pytorch_version": "2.2.0a0+git12b2dd1",
"device": "NVIDIA A100-SXM4-40GB",
"git_commit_hash": "12b2dd16b050e6495910fc564517fbb51dde1f20"
},
"metrics": {
"model=llama_v2_7b_16h, test=eval, device=cuda, bs=None, extra_args=['--accuracy'], metric=accuracy": "eager_1st_run_fail"
}
}
}
]
}
```
The bisection is successful.
Reviewed By: aaronenyeshi
Differential Revision: D51569643
Pulled By: xuzhao9
fbshipit-source-id: bb2bff715ba93807eb6c1549de540aeae364bf1e
