Run performance test non-alternately (#130136)
Summary:
By default, performance tests (speedup experiments) will run the baseline and test backend alternately.
However, this does not work for the torchao backend, which will change the model in-place, therefore the baseline run will also run with torchao backend since the model has already been quantized.
Add a new experiment "latency_experiment" to run performance tests non-alternately (first run baseline for a few iterations, then run the test backend).
X-link: https://github.com/pytorch/pytorch/pull/130136
Approved by: https://github.com/jerryzh168
Reviewed By: atalman
Differential Revision: D59332736
Pulled By: xuzhao9
fbshipit-source-id: 7dbd53bdf1a5d9bbc5bc704f2fabbdcff8c573ca
