Detect regression based on dates (#1609)
Summary:
Support using `--start_date` and `--end_date` to detect regressions by using files on Amazon S3.
Test plan (Need to set AWS credentials):
```
python regression_detector.py --platform gcp_a100 --end-date 2023-05-02 --name torch-nightly
```
```
control_env:
pytorch_git_version: 521b386e57f357386caa821adb0d9bae140f272e
treatment_env:
pytorch_git_version: f38a04a5c5ce632ba8b0534e3689e89f98e89d73
bisection: null
details:
test_eval[phlippe_resnet-cuda-eager]_latency:
control: 2.978329
treatment: 3.53091
delta: 0.18553390172811668
test_eval[timm_resnest-cuda-eager]_gmem:
control: 4.0126953125
treatment: 4.5927734375
delta: 0.14456072036991968
control_only_metrics: {}
treatment_only_metrics: {}
```
```
TorchBench CI has detected a performance signal or runtime regression.
Base PyTorch commit: 521b386e57f357386caa821adb0d9bae140f272e
Affected PyTorch commit: f38a04a5c5ce632ba8b0534e3689e89f98e89d73
Affected Tests:
- test_eval[phlippe_resnet-cuda-eager]_latency: +0.18553%
- test_eval[timm_resnest-cuda-eager]_gmem: +0.14456%
Tests that were no longer run on affected commit:
Tests that were newly added on affected commit:
Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!
GitHub workflow that triggered this issue: No URL found, please look for the failing action in https://github.com/pytorch/benchmark/actions
cc xuzhao9
```
Pull Request resolved: https://github.com/pytorch/benchmark/pull/1609
Reviewed By: janeyx99
Differential Revision: D45484697
Pulled By: xuzhao9
fbshipit-source-id: cd0b902c6f09bbf813f0e1c8fac07f6f148e61dd
