[fix bug] logits's shape different from label's shape in preprocess_logits_for_metrics (#31447)
* [fix BUG] pad labels before use it in preprocess_logits_for_metrics
* a more readable fix
labels can't use `gather` before pass to `preprocess_logits_for_metrics`, so must split into 2 if-block
* add a comment
* oh code quality check