Dynamic weight conversion is recursive (#44300)
* split out from timm PR
* all other VLMs
* timm backbone is not here
* oops, extra key is breaking eveerything
* .
* this test
* maybe
* fix missing keys when loading from hub
* now fix fast tests
* merge gone wrong
* fix repo
* refine the regex again!
* close the bracket
* Apply suggestions from code review
Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com>
* revert unrelated
* !
* revert more
* add submodule prefix when recursing
* i'll need to fix maskformer later
* dont duplicate the same pattern twice
* fix modular
* detr
* colpali isn't working still!
* oke, so this can be fine for now
* !
* revert
* dot lost in regex and comments
* timm wrapper is weird
* skip these, timm wrapper
* bye bye timm
* make repo check happy
* Revert "bye bye timm"
This reverts commit ca68663b20d056e5c7d490da0f8ff6ff9d597a3b.
* love timm!
* Apply repo consistency fixes
* oke, the bot can't fix it so here we go
---------
Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
