pytorch

39d89e06 - Upload test times to S3 (#49190)

Upload test times to S3 (#49190) Summary: This PR currently just modifies the `test/print_test_stats.py` script (run in the `pytorch_linux_test` job) so that now it uploads test times to the new `ossci-metrics` S3 bucket (rather than just to Scribe) if passed the `--upload-to-s3` parameter. The next step is to add an additional step to that `pytorch_linux_test` job which checks if it's being run on a PR, and if so, finds the `master` commit to compare against (similar to what's done in the now-unused `.jenkins/pytorch/short-perf-test-{c,g}pu.sh` scripts) and adds test time info to the Dr CI comment if the PR is significantly different from the base revision. Pull Request resolved: https://github.com/pytorch/pytorch/pull/49190 Test Plan: An "integration test" would be to just look in [the `ossci-metrics` S3 bucket](https://s3.console.aws.amazon.com/s3/buckets/ossci-metrics) to confirm that the CI run(s) for this PR did indeed upload their test time data successfully. To test this locally, first make sure you have all the packages you need, such as these: ``` $ conda install -c anaconda boto3 $ conda install -c conda-forge unittest-xml-reporting ``` Then run whatever tests you want; these are the ones I used for my local smoke test, for no particular reason: ``` $ python test/test_spectral_ops.py --save-xml=/tmp/reports/spectral_ops ``` Once the tests finish, run the script to upload their times to S3: ``` $ CIRCLE_SHA1="$(git rev-parse HEAD)" CIRCLE_JOB=foo test/print_test_stats.py --upload-to-s3 /tmp/reports/spectral_ops ``` Now check that they uploaded successfully: ``` $ aws s3 cp "s3://ossci-metrics/test_time/$(git rev-parse HEAD)/foo/" /tmp/reports --recursive ``` And that it's a valid `*.json.bz2` file: ``` $ bzip2 -kdc /tmp/reports/*Z.json.bz2 | jq . | head -n21 { "build_pr": null, "build_tag": null, "build_sha1": "e46f43621b910bc2f18dd33c08f5af18a542d5ed", "build_branch": null, "build_job": "foo", "build_workflow_id": null, "total_seconds": 0.9640000000000003, "suites": { "TestFFTCPU": { "total_seconds": 0.9640000000000003, "cases": [ { "name": "test_fft_invalid_dtypes_cpu", "seconds": 0.022, "errored": false, "failed": false, "skipped": false }, { "name": "test_istft_throws_cpu", ``` Reviewed By: walterddr, malfet Differential Revision: D25618035 Pulled By: samestep fbshipit-source-id: 4d8013859a38a49e5bba700c5134951ca1a9d8b7