Adding check for a single batch in adaptive_avg_pool
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/23137
Test Plan: Imported from OSS
Differential Revision: D16403804
Pulled By: zafartahirov
fbshipit-source-id: df79a8c768ffabeceb4c0044c967a623c5885484
