benchmark
a18bc489 - skip three pyhpc models with dynamic shape test (#120599)

Commit
1 year ago
skip three pyhpc models with dynamic shape test (#120599) Summary: As reported in https://github.com/pytorch/pytorch/issues/119434, `pyhpc_isoneutral_mixing`, `pyhpc_equation_of_state` and `pyhpc_turbulent_kinetic_energy` failed with dynamic shape testing, we propose to skip the dynamic batch size testing of these 3 models in this PR. * Error msg is ``` File "/localdisk/leslie/torch_inductor_community/pytorch/benchmarks/dynamo/common.py", line 3879, in run assert marked, f"nothing in example_inputs had a dim with {batch_size}" AssertionError: nothing in example_inputs had a dim with 1048576 ``` * Root Cause is * Benchmark code will only annotate the inputs' dim as dynamic when its size equals to batch size https://github.com/pytorch/pytorch/blob/c617e7b4076a5f968f5827040a07b013e45cd0c6/benchmarks/dynamo/common.py#L3867-L3871. If it fails to find any dim equals to batch size, above error throws. * However, for these 3 models, none of the inputs' dim will equal to input batch size since the [relationship of dim sizes](https://github.com/pytorch/benchmark/blob/26b85eadde28645c9b04b2d5a5b37f4d810b5100/torchbenchmark/models/pyhpc_equation_of_state/__init__.py#L12-L16) ``` shape = ( math.ceil(2 * size ** (1/3)), math.ceil(2 * size ** (1/3)), math.ceil(0.25 * size ** (1/3)), ) ``` * Another thing is `pyhpc_isoneutral_mixing`, `pyhpc_equation_of_state` can pass the dynamic batch size accuracy testing, because the batch size has been set to 4 in accuracy testing (https://github.com/pytorch/pytorch/blob/c617e7b4076a5f968f5827040a07b013e45cd0c6/benchmarks/dynamo/common.py#L3456) and `math.ceil(2 * size ** (1/3))` happens equaling to 4. * Since the dim sizes of input has above relationship, running the these models in dynamic shape, we may need to annotate `dim[0](s0) = dim[2](s1) * 8`, per the discussion in https://github.com/pytorch/pytorch/issues/117477#issuecomment-1897108756 avikchaudhuri, looks like we are not expressible for this case. So, I think we may need to skip the dynamic batch size testing for these 3 models. X-link: https://github.com/pytorch/pytorch/pull/120599 Approved by: https://github.com/jgong5, https://github.com/desertfire Reviewed By: kit1980 Differential Revision: D54343438 fbshipit-source-id: 69b2c2cac7e63f1bb3c6a4a69a22b203cae44d79
Parents
Loading