pytorch

8c64bb4f - [dper3] replace LengthsGather lowlevel module's PT implemetnatio to use caffe2 op

[dper3] replace LengthsGather lowlevel module's PT implemetnatio to use caffe2 op Summary: Use a more efficient C++ implementation in a caffe2 op to get rid of control flow statements here. Test Plan: - Ran `buck test dper3/dper3/modules/low_level_modules/tests:single_operators_test` - Ran `buck-out/gen/dper3/dper3_models/experimental/pytorch/ads_model_generation_script.par --model_type="inline_cvr_post_imp" --model_version="april_2020" --gen_inference_model` and observed files getting generated: ``` [ashenoy@devbig086.ash8 ~/fbsource/fbcode] ls -l /tmp/ashenoy/inline_cvr_post_imp_april_2020/ total 278332 -rw-r--r--. 1 ashenoy users 71376941 Sep 3 23:10 serialized_inline_cvr_post_imp_april_2020_model_inference.pt -rw-r--r--. 1 ashenoy users 71437424 Sep 3 22:09 serialized_inline_cvr_post_imp_april_2020_model_inference_shrunk.pt -rw-r--r--. 1 ashenoy users 14952 Sep 3 22:38 serialized_inline_cvr_post_imp_april_2020_model_io_metadata_map.pt -rw-r--r--. 1 ashenoy users 14952 Sep 3 21:42 serialized_inline_cvr_post_imp_april_2020_model_io_metadata_map_shrunk.pt -rw-r--r--. 1 ashenoy users 67001662 Sep 3 22:38 serialized_inline_cvr_post_imp_april_2020_model_main.pt -rw-r--r--. 1 ashenoy users 67126415 Sep 3 21:42 serialized_inline_cvr_post_imp_april_2020_model_main_shrunk.pt -rw-r--r--. 1 ashenoy users 3945257 Sep 3 22:34 serialized_inline_cvr_post_imp_april_2020_model_preproc.pt -rw-r--r--. 1 ashenoy users 4077266 Sep 3 21:37 serialized_inline_cvr_post_imp_april_2020_model_preproc_shrunk.pt ``` - Ran `buck-out/gen/dper3/dper3_models/experimental/pytorch/ads_model_generation_script.par --model_type="ctr_mbl_feed" --model_version="april_2020" --gen_inference_model` and observed model files getting generated: ``` [ashenoy@devbig086.ash8 ~/fbsource/fbcode] ls -l /tmp/ashenoy/ctr_mbl_feed_april_2020/ total 170304 -rw-r--r--. 1 ashenoy users 2641870 Sep 3 23:06 ctr_mbl_feed_april_2020_prod_eval_training_options -rw-r--r--. 1 ashenoy users 2641870 Sep 3 23:06 ctr_mbl_feed_april_2020_prod_train_training_options -rw-r--r--. 1 ashenoy users 42225079 Sep 3 23:59 serialized_ctr_mbl_feed_april_2020_model_inference.pt -rw-r--r--. 1 ashenoy users 42576708 Sep 3 22:33 serialized_ctr_mbl_feed_april_2020_model_inference_shrunk.pt -rw-r--r--. 1 ashenoy users 11194 Sep 3 23:29 serialized_ctr_mbl_feed_april_2020_model_io_metadata_map.pt -rw-r--r--. 1 ashenoy users 11194 Sep 3 22:05 serialized_ctr_mbl_feed_april_2020_model_io_metadata_map_shrunk.pt -rw-r--r--. 1 ashenoy users 39239139 Sep 3 23:29 serialized_ctr_mbl_feed_april_2020_model_main.pt -rw-r--r--. 1 ashenoy users 39250842 Sep 3 22:05 serialized_ctr_mbl_feed_april_2020_model_main_shrunk.pt -rw-r--r--. 1 ashenoy users 2839097 Sep 3 23:24 serialized_ctr_mbl_feed_april_2020_model_preproc.pt -rw-r--r--. 1 ashenoy users 2944239 Sep 3 22:01 serialized_ctr_mbl_feed_april_2020_model_preproc_shrunk.pt ``` Reviewed By: houseroad Differential Revision: D23519521 fbshipit-source-id: ed9bd16a8af3cca3a865d9614d67d07f01d8b18a