Remove spurious assert (#81604)
This assert was triggering after https://github.com/pytorch/functorch/pull/935
in some of the exhaustive python key tests, but the test is fine
even without it. I added an explanation in code.
Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/81604
Approved by: https://github.com/Chillee
