[OneFormer] Fix loss computation when is_training=False (#50039)
* Fix OneFormer loss crash when is_training=False
Skip the contrastive loss when there are no text queries: the text mapper
only exists with is_training=True, but released checkpoints set
contrastive_temperature with is_training=False.
* simplify
---------
Co-authored-by: Pablo Montalvo <pablo.montalvo.leroux@gmail.com>
Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com>