Restrict conversion to SmallVector (#42694)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/42694
The old implementation allowed calling SmallVector constructor and operator= for any type without restrictions,
but then failed with a compiler error when the type wasn't a collection.
Instead, we should only use it if Container follows a container concept and just not match the constructor otherwise.
This fixes an issue kimishpatel was running into.
ghstack-source-id: 109370513
Test Plan: unit tests
Reviewed By: kimishpatel, ezyang
Differential Revision: D22983020
fbshipit-source-id: c31264f5c393762d822f3d64dd2a8e3279d8da44