Grounding dino fp16 dtype [backlog] (#48438)
* align dtype of queries, keys in Grounding DINO
To support #32672
* update dtype of grid_y, grid_x
* Create dtype-dependent tensors at the model dtype in Grounding DINO
valid_ratios and the reference point grid were built as float32 regardless of
the model dtype, so grid_sample and the reference point head received float32
against half weights. fp16 and bf16 both crashed.
get_valid_ratio gains the dtype parameter that deformable_detr, lw_detr,
mask2former, oneformer and rf_detr already have.
Fixes #32672
Co-authored-by: Sam <samhk1807@gmail.com>
---------
Co-authored-by: Sam <samhk1807@gmail.com>
