benchmark

3f2a2a15 - Set threshold to 30% and disable torch-nightly unstable models (#1681)

Set threshold to 30% and disable torch-nightly unstable models (#1681) Summary: Skip models that are deemed unstable by torch-nightly entirely. Currently, that would be the following 16: ``` ['LearningToPaint', 'densenet121', 'detectron2_fasterrcnn_r_101_c4', 'detectron2_fasterrcnn_r_101_fpn', 'detectron2_fasterrcnn_r_50_c4', 'detectron2_fasterrcnn_r_50_c4', 'detectron2_fcos_r_50_fpn', 'detectron2_maskrcnn_r_50_c4', 'detectron2_maskrcnn_r_50_fpn', 'drq', 'functorch_dp_cifar10', 'functorch_maml_omniglot', 'lennard_jones', 'resnext50_32x4d', 'soft_actor_critic', 'timm_efficientdet'] ``` Also drive the regression threshold to 30%!!! This _is_ a big leap, but we want to stop flooding torchbench with issues everyday and slowly push down the threshold. Based on stats in https://docs.google.com/spreadsheets/d/1r3a7UczDofej8p6KhAcbTf8moxnlbyQ1AWZKOo0kbh8/edit#gid=418753160, only 0.8% of noise will go beyond 30%, so this should make the issues much more manageable in conjunction with my nifty dashboard (unfortunately internal only atm) www.fburl.com/optim_benchmarks Pull Request resolved: https://github.com/pytorch/benchmark/pull/1681 Reviewed By: albanD, xuzhao9 Differential Revision: D46121268 Pulled By: janeyx99 fbshipit-source-id: 16b4dffbe2e82256aa3fb0229b9626e61aa0789e