Fix the broken CI by disabling warmup by default (#1582)
Summary:
We should not run warmup rounds to get eager latency when eager mode is already used.
However, in non-eager mode, we need to do the following:
1. Run correctness rounds with fp32 precision to get accurate eager output
2. Run warmup rounds with default precision to get accurate eager latency
3. Run dynamo rounds to get the correctness result
Note that the warmup rounds here are different from the warmup iterations in `run.py`. Here, we use a few warmup rounds to approximate eager latency, which is later used to calculate PT2 compilation time. In `run.py`, we run warmup rounds for performance testing.
Fix https://github.com/pytorch/benchmark/issues/1581
Pull Request resolved: https://github.com/pytorch/benchmark/pull/1582
Reviewed By: janeyx99
Differential Revision: D45327010
Pulled By: xuzhao9
fbshipit-source-id: 56bdf9966df99cea9d4de26cebe704b96b34d56d
