pytorch

0d7036fd - don't leak build time path name to runtime for frozen python modules (#65715)

don't leak build time path name to runtime for frozen python modules (#65715) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/65715 Here is how we freeze a python module: - we call python builtin compile method with the source code of the modules and the path. This method returns a python code object - we call marshal.dumps to serialize the code object to bytes. The code_object.co_filename actually matches the one passed in to the compile method. We can simply replace that with a marker to avoid leak build time path to runtime. This works on nested code objects as well: ``` #!/bin/env python3.8 import marshal code_str = """ print("hello") class MyCls: def __init__(self): pass """ co = compile(code_str, "<Generated by torch::deploy>", "exec") cobytes = marshal.dumps(co) import pdb; pdb.set_trace() ``` Checking `co`: ``` (Pdb) co.co_filename '<Generated by torch::deploy>' (Pdb) co.co_consts ('hello', <code object MyCls at 0x7f0e8670bbe0, file "<Generated by torch::deploy>", line 4>, 'MyCls', None) (Pdb) co.co_consts[1].co_filename '<Generated by torch::deploy>' ``` Test Plan: Find the serialized frozenmodule for torch.nn.modules.linear module in the generated bytecode_x.c file. Put the content to /tmp/linear.bytecode Run the testing script: ``` import marshal co_bytes = bytes(eval("[{}]".format("".join(open('/tmp/linear.bytecode').readlines()).replace('\n', '').replace('\t', '')))) co = marshal.loads(co_bytes) print(co) ``` The output for the paste without the change: ``` <code object <module> at 0x7f39ca7f07c0, file "/data/users/shunting/fbsource/fbcode/buck-out/opt/gen/caffe2/gen_frozen_torchpython_src__srcs/torch/nn/modules/linear.py", line 1> ``` The output for the paste with the change: ``` <code object <module> at 0x7f05a765d710, file "<Generated by torch::deploy>", line 1> ```` Note that the file part is changed as expected. Reviewed By: suo Differential Revision: D31214555 fbshipit-source-id: 56958e0a7352f8c30a3377f83209efe7db61f0fb