Add new dim to `num_items_in_batch` if necessary (#36967)
* Add new dim to `num_items_in_batch` if necessary
* Unsqueeze only in the DP case
---------
Co-authored-by: Ilyas Moutawwakil <57442720+IlyasMoutawwakil@users.noreply.github.com>
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>