Move RefInfo classes into opinfo.refs (#83563)
Given that there is already a clear `op_db`, `python_ref_db` split I
think it makes sense to have the `RefInfo` classes be defined in a
different file.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/83563
Approved by: https://github.com/albanD
