Add AtenOp Benchmarking (#2495)
Summary:
As described in https://github.com/pytorch/pytorch/issues/136168, I'm trying to migrate native PyTorch implementation comparison([the original operatorbench](https://github.com/pytorch/pytorch/blob/main/benchmarks/dynamo/microbenchmarks/operatorbench.py)) to TritonBench.
This PR adds an Operator Loader which can load aten ops used in TorchBench, HuggingFace, and TIMM models. The benchmark classes are dynamically created. Then benchmark them between aten and inductor implementations.
Files `torchbenchmark/operator_loader/operator_inp_utils.py`, `torchbenchmark/operator_loader/operatorbench.py`, and all configs files in `torchbenchmark/operator_loader/operator_inp_logs/` are copied from original operatorbench.
Example commands:
```bash
python run_benchmark.py triton --op aten._softmax.default --num-inputs 1 --operator-loader --precision fp16
```
Exampled Output:
```
Evaluating an op name into an OpOverload: The underlying op of 'aten.upsample_nearest2d_backward' has no overload name 'vec'
Evaluating an op name into an OpOverload: '_OpNamespace' 'aten' object has no attribute 'im2col_backward'
Evaluating an op name into an OpOverload: '_OpNamespace' 'aten' object has no attribute 'col2im_backward'
Evaluating an op name into an OpOverload: '_OpNamespace' 'aten' object has no attribute 'im2col_backward'
Evaluating an op name into an OpOverload: The underlying op of 'aten.upsample_bilinear2d_backward' has no overload name 'vec'
Evaluating an op name into an OpOverload: The underlying op of 'aten.upsample_nearest2d_backward' has no overload name 'vec'
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:02<00:00, 1.20s/it]
x_val eager-latency inductor-latency
------- --------------- ------------------
0 0.090592 0.089632
1 0.055808 0.038112
```
Pull Request resolved: https://github.com/pytorch/benchmark/pull/2495
Reviewed By: xuzhao9
Differential Revision: D64200358
Pulled By: FindHao
fbshipit-source-id: f0121168b33247224bc905a1a88af69e4b13def6
