Revert "Simplify handle indexing (#105006)" (#105984)
This reverts commit 429d45f91a5b636844954363851be309d8203b56.
Unfortunately, https://github.com/pytorch/pytorch/pull/105006 broke backward prefetching (where backward prefetching working correctly was not captured in our unit tests).
I need more time to dig into this (tomorrow), but I think the issue is related to:
https://github.com/pytorch/pytorch/commit/429d45f91a5b636844954363851be309d8203b56#diff-9a6937168d232432c34c2c4605b96f3147afa2786e287f74b6074b20aa5980e6R143-R146
Follow-ups:
1. Investigate this thoroughly
2. Add unit tests to capture backward prefetch functionality
Pull Request resolved: https://github.com/pytorch/pytorch/pull/105984
Approved by: https://github.com/fegin
