functorch userbenchmark (#1235)
Summary:
functorch userbenchmark
This PR sets up the functorch userbenchmark and configures it to run 3
functorch models in the torchbenchmark suite that have not made it to
stable (e.g. torchbench v3) yet, so we can get regression testing on
them.
In addition, this PR adds some microbenchmarks for functorch into the userbenchmark.
Pull Request resolved: https://github.com/pytorch/benchmark/pull/1235
Test Plan:
`python run_benchmark.py functorch` gives:
```
(/raid/rzou/pt/binaries4-env) [1] rzou@devfair0317:~/pytorch/benchmark (functorch_userbenchmark) $ cat .userbenchmark/func
torch/metrics-20221007144500.json
{
"name": "functorch",
"environ": {
"git_version": "b472915d42b840469910e8fb243d40441a4b5a17"
},
"metrics": {
"lennard_jones_cpu": 4.601219408214092,
"lennard_jones_cuda": 6.164539773017168,
"functorch_maml_omniglot_cpu": 12597.586147487164,
"functorch_maml_omniglot_cuda": 318.9176507294178,
"functorch_dp_cifar10_cuda": 83.94835330545902,
"vmap_hessian_fc_cuda": 60.61760187149048,
"SimpleMLP_vmap_cpu": 0.46687781251966953,
"SimpleMLP_vmap_cuda": 0.36797603964805603,
"SimpleMLP_ensemble_multi_cpu": 2.3521081916987896,
"SimpleMLP_ensemble_multi_cuda": 0.502417491748929,
"SimpleCNN_ensemble_multi_cuda": 5.7795668579638,
"SimpleMLP_ensemble_single_cpu": 2.3946920223534107,
"SimpleMLP_ensemble_single_cuda": 0.5035349484533072,
"SimpleCNN_ensemble_single_cuda": 5.614033993333578,
"SimpleMLP_persamplegrad_cpu": 8.035034853965044,
"SimpleMLP_persamplegrad_cuda": 1.9205754343420267,
"SimpleCNN_persamplegrad_cuda": 3.8312537409365177
}
}
```
Reviewed By: xuzhao9
Differential Revision: D40196270
Pulled By: xuzhao9
fbshipit-source-id: 45cc7a8631fd460f98458e3b215e0fd8fedc14d7
