transformers
56c5e876 - Fix silently random-initializing `RTDetrModel`/`SEWDForCTC` loads (wrong `base_model_prefix`) (#48744)

Commit
67 days ago
Fix silently random-initializing `RTDetrModel`/`SEWDForCTC` loads (wrong `base_model_prefix`) (#48744) * Fix `base_model_prefix` for RT-DETR family and SEW-D: silently random-initialized loads `RTDetrModel.from_pretrained("PekingU/rtdetr_r18vd")` and `SEWDForCTC.from_pretrained("asapp/sew-d-tiny-100k")` silently returned a fully randomly-initialized model (502/502 and 222/222 missing keys respectively), because the weight loader adds/strips `base_model_prefix` when crossing the base-model/head-model boundary, and for these families the declared prefix never matches the actual attribute: - `RTDetrForObjectDetection` nests the base model in `self.model`, but the prefix was `"rt_detr"` (materialized as `"rt_detr_v2"`, `"pp_doclayout_v2"`, `"pp_doclayout_v3"`, `"d_fine"`, `"deimv2"` for the subclasses through the modular converter). Detection checkpoints store every weight under `model.`, which was never stripped when loading the base `*Model` classes. - `SEWDForCTC` nests the base model in `self.sew_d` while the prefix was `"sew-d"` (with a dash), which can never match a Python attribute name, so base checkpoints (unprefixed keys) never got the `sew_d.` prefix added. Align the prefix with the actual attribute (`"model"` in the RT-DETR family, `"sew_d"` for SEW-D), matching what sibling architectures already do (`DeformableDetr`/`TableTransformer`/`RfDetr` use `"model"`; `SEW` uses `"sew"`). For the RT-DETR family the fix lives in `modular_rt_detr.py` and is propagated to all generated modeling files. This also un-breaks `model.base_model` for these classes (it used to return the head model itself), so the generic `test_correct_missing_keys` (`test_missing_keys = False`) is re-enabled for rt_detr, rt_detr_v2, pp_doclayout_v2, pp_doclayout_v3, d_fine and deimv2. Add cross-class save/load regression tests for all seven families, plus hub-level tests for the exact checkpoints reported in the issue. Fixes #48722 * no need for new test --------- Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
Author
Parents
Loading