pytorch

3c5677a6 - Use codegen'ed unboxing wrappers (#32521)

Use codegen'ed unboxing wrappers (#32521) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/32521 Not all ops support the templated unboxing wrappers yet. For the ones that don't, let's use the codegen'ed unboxing wrappers from register_aten_ops.cpp, but register them with c10 directly instead of JIT. The `use_c10_dispatcher` setting in `native_functions.yaml` now has a new option 'with_codegenerated_unboxing_wrapper' which means we take the codegened unboxing wrapper from register_aten_ops.cpp and stuff it into c10. This new argument is the default, 'unboxed_only' is not the default anymore. For the (very few) ops that don't support boxed dispatch yet (i.e. ops taking TensorOptions arguments), we set them to 'unboxed_only' and they follow the old behavior of having register_aten_ops.cpp register the jit op. Next steps here are (1) to make TensorOptions work with boxed dispatch and remove the `unboxed_only` option from `use_c10_dispatcher`, so that all ops go through the new path and (2) make the new path template-only and remove codegen from it (see https://github.com/pytorch/pytorch/issues/32366). First experiments show that - For a small JITted model that calls add (i.e. a op with just two arguments that are both tensors) on two tensors in a loop, we see a 2-4% performance improvement (~35-50ns) when compared to the old path. This is a simple op that takes two tensor arguments and no non-tensor arguments, so iterating over it in boxed dispatch is cheap. - For a small JITted model that calls avgpool1d (i.e. an op that has one tensor arg and 5 non-tensor args) on a tensor in a loop, we see a 3-4% performance regression (~60ns) when compared to the old path. This is an op that takes only one tensor argument and then 6 non-tensor arguments. Unboxed dispatch doesn’t have to look at those but boxed dispatch still needs to iterate over them. This performance difference is likely due to boxed dispatch iterating over all arguments in a loop and unboxed dispatch not having to look at non-tensor arguments. ghstack-source-id: 99161484 Test Plan: unit tests that call existing ops through JIT Differential Revision: D18672405 fbshipit-source-id: bf2a7056082dfad61e7e83e9eeff337060eb6944