Revert "Always convert truthy booleans to 1"
This reverts commit 3c3c6cd9821dc48182cbfb96572cc562b76a375e.
Reverted https://github.com/pytorch/pytorch/pull/77122 on behalf of https://github.com/mruberry due to broke some jobs, like https://github.com/pytorch/pytorch/runs/6706333043?check_suite_focus=true
