pytorch
17217d36 - Inductor cpp wrapper: support Constant in input (#103496)

Commit
1 year ago
Inductor cpp wrapper: support Constant in input (#103496) ## Description Fix cpp wrapper for models which have constants in the graph inputs. Python wrapper directly gets the value inside the wrapper call as a global variable passed when calling: https://github.com/pytorch/pytorch/blob/4081e924a8d701d5201db3e4b4ed2da60b072d30/torch/_inductor/codecache.py#L757 The constants value has been saved in `mod.__dict__` in https://github.com/pytorch/pytorch/blob/4081e924a8d701d5201db3e4b4ed2da60b072d30/torch/_inductor/graph.py#L874-L875 For cpp wrapper, we need to append constants to the input args, so as to pass this python value to the `inductor_entry_cpp` function explicitly. ### Example Example of output code for dlrm in TorchBench with this fix: ```py module = CppWrapperCodeCache.load(cpp_wrapper_src, 'inductor_entry_cpp', 'cfkc6c36t7cggi6mnokrdm5jhesnunjg5xysv3o3x3vaqmzmpe6r', False) def _wrap_func(f): def g(args): args_tensor = [arg if isinstance(arg, torch.Tensor) else torch.tensor(arg) for arg in args] constants_tensor = [constant0, constant1] args_tensor.extend(constants_tensor) return f(args_tensor) return g call = _wrap_func(module.inductor_entry_cpp) ``` Pull Request resolved: https://github.com/pytorch/pytorch/pull/103496 Approved by: https://github.com/jgong5, https://github.com/jansel, https://github.com/desertfire
Author
Committer
Parents
Loading