add API coverage test (#1394)
Summary:
This PR adds the userbenchmark for PyTorch API coverage test
Command to run for a specific model:
```
python3 run_benchmark.py api-coverage --model resnet18 --device cuda --test train,eval
```
If you want to test all models, the `--model` should not be specified.
The testing results are stored in `.userbenchmark/api-coverage/logs/`. Each run will generate two log files, logs-${timestamp}.json and logs-${timestamp}.json-api_coverage.csv. The latter one is like the following format.
```
API coverage rate: 25/1332 = 1.88%
missed APIs:
module_name,func_name
,_TensorBase.acos
...
```
All missed APIs will be listed in the csv file.
If a logs-${timestamp}.json-api_need_support.csv is also generated, please open an issue to report the miss supported APIs by this userbenchmark.
Pull Request resolved: https://github.com/pytorch/benchmark/pull/1394
Reviewed By: xuzhao9
Differential Revision: D43737578
Pulled By: FindHao
fbshipit-source-id: c7570f395572e26dfce52529195e8575e03185ce
