Update internal invariants in the world of manuallyBoxedKernel (#36388)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/36388
(This wants to make me barf).
Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Test Plan: Imported from OSS
Differential Revision: D20964195
Pulled By: ezyang
fbshipit-source-id: 3699a02b16060d79dae9890bafeaafad9ad9ae60