benchmark

d0a05378 - Add feature to run nightly with regression yaml (#1677)

Add feature to run nightly with regression yaml (#1677) Summary: To enable bisection, the userbenchmark needs to run with a regression yaml file. This is an optimization: if there is only 1 test regresses, we should only run the regressed test as opposed to the entire test set. Pull Request resolved: https://github.com/pytorch/benchmark/pull/1677 Test Plan: ``` python run_benchmark.py torch-nightly --run-bisect ./.userbenchmark/torch-nightly/regression-20230522231642.yaml --dryrun ``` ``` Running TorchBenchModelConfig(name='BERT_pytorch', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='DALLE2_pytorch', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='LearningToPaint', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='Super_SloMo', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='alexnet', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='attention_is_all_you_need_pytorch', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='dcgan', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='demucs', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_fasterrcnn_r_101_dc5', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_fasterrcnn_r_50_dc5', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_fasterrcnn_r_50_fpn', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_maskrcnn', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_maskrcnn_r_101_c4', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_maskrcnn_r_101_fpn', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_maskrcnn_r_50_fpn', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='dlrm', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='doctr_det_predictor', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='doctr_reco_predictor', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='fambench_xlmr', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='fastNLP_Bert', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='functorch_dp_cifar10', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_Albert', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_Bart', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_Bert', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_Bert_large', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_BigBird', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_DistilBert', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_GPT2', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_GPT2_large', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_Longformer', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_Reformer', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_T5', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_T5_base', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_T5_large', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='llama', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='maml', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='maml_omniglot', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='mnasnet1_0', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='mobilenet_v2', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='mobilenet_v3_large', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='moco', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='nvidia_deeprecommender', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='opacus_cifar10', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='phlippe_densenet', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='phlippe_resnet', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='pyhpc_equation_of_state', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='pyhpc_isoneutral_mixing', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='pyhpc_turbulent_kinetic_energy', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='pytorch_CycleGAN_and_pix2pix', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='pytorch_stargan', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='pytorch_unet', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='resnet152', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='resnet18', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='resnet50', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='resnext50_32x4d', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='shufflenet_v2_x1_0', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='soft_actor_critic', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='speech_transformer', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='squeezenet1_1', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='tacotron2', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_efficientnet', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_nfnet', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_regnet', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_resnest', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_vision_transformer', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_vision_transformer_large', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_vovnet', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='torchrec_dlrm', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='tts_angular', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='vgg16', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='vision_maskrcnn', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='yolov3', device='cuda', test='eval', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='Background_Matting', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='DALLE2_pytorch', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='Super_SloMo', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='alexnet', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='attention_is_all_you_need_pytorch', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='dcgan', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='densenet121', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_fasterrcnn_r_101_c4', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_fasterrcnn_r_101_dc5', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_fasterrcnn_r_101_fpn', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_fasterrcnn_r_50_dc5', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_fasterrcnn_r_50_fpn', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_maskrcnn', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_maskrcnn_r_101_c4', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_maskrcnn_r_101_fpn', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='detectron2_maskrcnn_r_50_c4', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='dlrm', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='drq', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='fambench_xlmr', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='fastNLP_Bert', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='functorch_maml_omniglot', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_Albert', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_Bart', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_Bert', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_Bert_large', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_BigBird', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_DistilBert', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_GPT2', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_GPT2_large', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_Longformer', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_Reformer', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_T5', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='hf_T5_large', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='lennard_jones', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='maml_omniglot', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='mnasnet1_0', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='mobilenet_v2', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='mobilenet_v2_quantized_qat', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='mobilenet_v3_large', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='moco', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='nvidia_deeprecommender', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='opacus_cifar10', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='phlippe_densenet', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='phlippe_resnet', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='pytorch_CycleGAN_and_pix2pix', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='pytorch_stargan', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='pytorch_struct', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='pytorch_unet', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='resnet152', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='resnet18', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='resnet50', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='resnet50_quantized_qat', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='shufflenet_v2_x1_0', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='speech_transformer', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='squeezenet1_1', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='tacotron2', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_efficientdet', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_efficientnet', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_nfnet', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_regnet', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_resnest', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_vision_transformer', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='timm_vovnet', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='torchrec_dlrm', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='tts_angular', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='vgg16', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='vision_maskrcnn', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] Running TorchBenchModelConfig(name='yolov3', device='cuda', test='train', batch_size=None, jit=False, extra_args=[], extra_env=None) ... [Skip: Dryrun] ``` Reviewed By: janeyx99 Differential Revision: D46108449 Pulled By: xuzhao9 fbshipit-source-id: 22c8ba95c183f8902108a4c3c1a39cd35f24e2ac