Update unary_ufuncs groupings to include primtorch types. (#107345)
Fixes #107335. The skips were updated for the _ref ops to match those for eager mode where necessary. Part of breakdown of https://github.com/pytorch/pytorch/pull/104489.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/107345
Approved by: https://github.com/ezyang
