Fix some GPU OOM after #37553 (#37591)

Commit

285 days ago

Fix some GPU OOM after #37553 (#37591) * fix * trigger CI --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>