Refactor and improve randperm tests.
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/22121
Test Plan: Imported from OSS
Differential Revision: D16153794
Pulled By: li-roy
fbshipit-source-id: 4dbfa6cfcc79f6d431918a6646664215fa9ea0b9
