pytorch
16bd3638 - Fix dynamo dashboard passrate denominator (#88777)

Commit
2 years ago
Fix dynamo dashboard passrate denominator (#88777) Before the dashboard improvements, the passrate table looked like this: ~~~ +------------------------+------------+-------------+-------------+ | Compiler | torchbench | huggingface | timm_models | +------------------------+------------+-------------+-------------+ | eager | 98%, 54/55 | 100%, 43/43 | 100%, 61/61 | | aot_eager | 95%, 52/55 | 100%, 43/43 | 97%, 59/61 | | aot_cudagraphs | 75%, 41/55 | 49%, 21/43 | 38%, 23/61 | | nvprims_nvfuser | 71%, 39/55 | 16%, 7/43 | 48%, 29/61 | | inductor | 87%, 48/55 | 93%, 40/43 | 95%, 58/61 | | inductor_no_cudagraphs | 93%, 51/55 | 93%, 40/43 | 95%, 58/61 | +------------------------+------------+-------------+-------------+ ~~~ After the change, the table looked like: ~~~ +------------------------+------------+-------------+-------------+ | Compiler | torchbench | huggingface | timm_models | +------------------------+------------+-------------+-------------+ | eager | 82%, 53/65 | 84%, 43/51 | 82%, 61/74 | | aot_eager | 83%, 54/65 | 84%, 43/51 | 82%, 61/74 | | aot_cudagraphs | 69%, 45/65 | 65%, 33/51 | 38%, 28/74 | | nvprims_nvfuser | 48%, 31/65 | 78%, 40/51 | 26%, 19/74 | | inductor | 75%, 49/65 | 82%, 42/51 | 81%, 60/74 | | inductor_no_cudagraphs | 82%, 53/65 | 82%, 42/51 | 82%, 61/74 | +------------------------+------------+-------------+-------------+ ~~~ There is no actual regression, but the passrate is lower since the denominator is wrong. Check fix by running locally (e.g. `python benchmarks/dynamo/runner.py --output-dir ../test-dynamo-runner-logs-5 --training --visualize_logs`) and comparing passrate table output to previously correct one. Pull Request resolved: https://github.com/pytorch/pytorch/pull/88777 Approved by: https://github.com/anijain2305
Author
Committer
Parents
Loading