Remove numpy assert that fails on Windows (older numpy versions). (#24012)
Summary:
Fixes https://github.com/pytorch/pytorch/issues/24001.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/24012
Differential Revision: D16732191
Pulled By: ezyang
fbshipit-source-id: 36660a6635ab64d2f63278b1616deb1282dea037