[fix] polygamma n>=1 (#61641)
Summary:
Fixes: https://github.com/pytorch/pytorch/issues/55357
TODO:
* [x] Use proper casting to avoid confusing the compiler
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61641
Reviewed By: albanD
Differential Revision: D29816592
Pulled By: mruberry
fbshipit-source-id: 2c020a6e4c325c1b5d15499a77fb39f9ba93dd79