Fix paligemma inverted mask (#31207)

Commit

1 year ago

Fix paligemma inverted mask (#31207) * pass inverted causal mask * add sanity check for paligemma finetuning * [run-slow]paligemma