pytorch

5f9fb93c - [model loading] Add max_batch_size override for batch size exploration

[model loading] Add max_batch_size override for batch size exploration Summary: Currently batch_size is determined on modeling side. Add a flag caffe2_predictor_disagg_acc_max_batch_size_override to explore different batch_size during inference. Test Plan: replayer test set caffe2_predictor_disagg_acc_max_batch_size_override=32 on both server and client side. Reviewed By: khabinov Differential Revision: D26318568 fbshipit-source-id: 4fa79e2087a5f7f7670988aec7e5b41e63f9980b