Support skipping tests on a blacklist in test.py (#584)
Summary:
This allows us to control which tests are run on the CI
machines. Ex. we may introduce training batch sizes from
the original papers that are too large to fit on the CI
machine.
Before:
```
$ gpurun python test.py -k densenet --verbose
srun: job 8076 queued and waiting for resources
srun: job 8076 has been allocated resources
test_densenet121_check_device_cpu (__main__.TestBenchmark) ... ok
test_densenet121_check_device_cuda (__main__.TestBenchmark) ... ok
test_densenet121_eval_cpu (__main__.TestBenchmark) ... ok
test_densenet121_eval_cuda (__main__.TestBenchmark) ... ok
test_densenet121_example_cpu (__main__.TestBenchmark) ... ok
test_densenet121_example_cuda (__main__.TestBenchmark) ... ok
test_densenet121_train_cpu (__main__.TestBenchmark) ... ok
test_densenet121_train_cuda (__main__.TestBenchmark) ... ok
----------------------------------------------------------------------
Ran 8 tests in 115.684s
OK
```
After:
```
$ gpurun python test.py -k densenet --verbose
srun: job 8079 queued and waiting for resources
srun: job 8079 has been allocated resources
test_densenet121_check_device_cpu (__main__.TestBenchmark) ... ok
test_densenet121_check_device_cuda (__main__.TestBenchmark) ... ok
test_densenet121_eval_cpu (__main__.TestBenchmark) ... ok
test_densenet121_eval_cuda (__main__.TestBenchmark) ... ok
test_densenet121_example_cpu (__main__.TestBenchmark) ... ok
test_densenet121_example_cuda (__main__.TestBenchmark) ... ok
test_densenet121_train_cpu (__main__.TestBenchmark) ... ok
test_densenet121_train_cuda (__main__.TestBenchmark) ... skipped 'This test is on the blacklist'
----------------------------------------------------------------------
Ran 8 tests in 106.145s
OK (skipped=1)
```
Pull Request resolved: https://github.com/pytorch/benchmark/pull/584
Reviewed By: xuzhao9
Differential Revision: D32668545
Pulled By: aaronenyeshi
fbshipit-source-id: 4c8ae39c480384e9ab5a4476f98a9448f69ba639
