benchmark

ad061568 - Automatically detect anomaly, report and bisect it if detected (#488)

Automatically detect anomaly, report and bisect it if detected (#488) Summary: This PR adds the ability to automatically detect performance signals after each nightly run. It uses the threshold of `7%` to define a performance signal. That is to say, if any tests show a latency change of > 7%, we will treat it as a performance signal and do the following: 1. Start a bisection job 2. Create a pytorch/benchmark GitHub issue so that we can keep track of the bisection result and assign it to the right owner Pull Request resolved: https://github.com/pytorch/benchmark/pull/488 Test Plan: V1 Nightly CI Reviewed By: aaronenyeshi Differential Revision: D31768545 Pulled By: xuzhao9 fbshipit-source-id: e099ad6b98a832a963fbdb533dcb5bfd5a0ca0f3