pytorch

8e51381b - Make AOT compiler generic (#68637)

Make AOT compiler generic (#68637) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/68637 Make AOT compiler compile BI bytedoc model also making the compiler generic enough for other models. Shape propagation pass replaced with the new JIT tracer as shape propagation doesn't yet support dynamic shapes. Change to get and set input dtype to follow Test Plan: BI model changed to return a tuple of tensors instead of returning a tuple(list[tensor], list[string]). Modified BI model runs well with these changes ``` jf download GN91Hg9shoWzU1oPAGQ7X9SV8-5nbmQwAAAA --file bi.pt └─ $ ./compile_model.sh -m pytorch_dev_bytedoc -p bi.pt -v v1 -i "1,115;1" + VERSION=v1 + getopts m:p:v:i:h opt + case $opt in + MODEL=pytorch_dev_bytedoc + getopts m:p:v:i:h opt + case $opt in + MODEL_PATH=bi.pt + getopts m:p:v:i:h opt + case $opt in + VERSION=v1 + getopts m:p:v:i:h opt + case $opt in + INPUT_DIMS='1,115;1' + getopts m:p:v:i:h opt + require_arg m pytorch_dev_bytedoc + '[' -n pytorch_dev_bytedoc ']' + require_arg p bi.pt + '[' -n bi.pt ']' + require_arg i '1,115;1' + '[' -n '1,115;1' ']' + '[' '!' -f bi.pt ']' +++ dirname ./compile_model.sh ++ cd . ++ pwd -P + SRC_DIR=/data/users/priyaramani/fbsource/fbcode/caffe2/fb/nnc + FBCODE_DIR=/data/users/priyaramani/fbsource/fbcode/caffe2/fb/nnc/../../.. + FBSOURCE_DIR=/data/users/priyaramani/fbsource/fbcode/caffe2/fb/nnc/../../../.. + KERNEL_DIR=/data/users/priyaramani/fbsource/fbcode/caffe2/fb/nnc/../../../../xplat/pytorch_models/build/pytorch_dev_bytedoc/v1/nnc ++ readlink -f bi.pt ++ sed 's/.pt.*//' + MODEL_PATH_PREFIX=/data/users/priyaramani/fbsource/fbcode/caffe2/fb/nnc/bi + LLVM_CODE_PATH=/data/users/priyaramani/fbsource/fbcode/caffe2/fb/nnc/bi.compiled.ll + ASSEMBLY_CODE_PATH=/data/users/priyaramani/fbsource/fbcode/caffe2/fb/nnc/bi.compiled.s + COMPILED_MODEL_FILE_PATH=/data/users/priyaramani/fbsource/fbcode/caffe2/fb/nnc/bi.compiled.pt + KERNEL_FUNC_NAME=nnc_pytorch_dev_bytedoc_v1_forward + buck run //caffe2/binaries:aot_model_compiler -- --model=bi.pt --model_name=pytorch_dev_bytedoc --model_version=v1 '--input_dims=1,115;1' Restarting Buck daemon because Buck version has changed... Buck daemon started. Parsing buck files... 0.6 sec (0/unknown) . . Parsing buck files: finished in 5.0 sec Creating action graph: finished in 0.7 sec Downloaded 3750/4917 artifacts, 16.09 Mbytes, 13.3% cache miss (for updated rules) Building: finished in 01:22.3 min (100%) 4995/4995 jobs, 4995/4995 updated Total time: 01:28.0 min BUILD SUCCEEDED Run with 56 threads Run with 56 threads Loading model... Model loaded: /data/users/priyaramani/fbsource/fbcode/caffe2/fb/nnc/bi.compiled.pt Running forward ... WARNING: Logging before InitGoogleLogging() is written to STDERR W1115 11:42:18.170666 1597103 TensorImpl.h:1418] Warning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (function operator()) (Columns 1 to 10 0.5428 0.1651 0.0158 0.0055 0.0503 0.0749 0.0161 0.0204 0.0237 0.0095 Columns 11 to 12 0.0609 0.0148 [ CPUFloatType{1,12} ], Columns 1 to 10-1.3946 -0.0835 -1.1268 0.3325 -2.1884 4.6175 -0.1206 -1.5058 -1.5277 -2.1214 Columns 11 to 20 1.3726 -0.4573 -1.7583 -2.2275 1.9607 -5.3430 -4.4927 -3.2548 -5.3214 2.9002 Columns 21 to 30-1.3973 -0.8084 -1.8491 -1.6518 4.2531 -0.0321 -0.0282 -1.1180 -0.9800 2.9228 Columns 31 to 32 0.8228 2.2611 [ CPUFloatType{1,32} ]) Starting benchmark. Running warmup runs. Main runs. Main run finished. Milliseconds per iter: 40.64. Iters per second: 24.6063 Memory usage before main runs: 71581696 bytes Memory usage after main runs: 94347264 bytes Peak memory usage after main runs: 94347264 bytes Average memory increase per iter: 2.22495e+06 bytes 0 value means "not available" in above ``` Reviewed By: ljk53 Differential Revision: D32438852 fbshipit-source-id: 5defdc2593abda5da328f96248459d23b2c5e5c6