clean up binaries/cmake for mobile (#25651)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/25651
Most of the bianries are not useful/compilable for mobile. Consolidate the gating
logic and move to the beginning of the file.
Test Plan: - make sure BUILD_BINARY=ON works for both mobile and non-mobile builds;
Differential Revision: D17183550
Pulled By: ljk53
fbshipit-source-id: a8179f4e80999271bf43b5d97798abc713c59843
