Stop using my channel for 3.11 builds (#96973)
As `numpy` for Python 3.11 is now available from the default anaconda channel
Pull Request resolved: https://github.com/pytorch/pytorch/pull/96973
Approved by: https://github.com/kit1980, https://github.com/atalman
