transformers

54104656 - Fix tokenizer auto_map being ignored for custom models (#43219)

Fix tokenizer auto_map being ignored for custom models (#43219) * Fix tokenizer auto_map being ignored for custom models (#43202) PR #42894 added an early-exit to TokenizersBackend when tokenizer_class doesn't match the registered tokenizer for a model_type. However, this early-exit was placed before the auto_map check, causing custom tokenizers with trust_remote_code to be ignored. This fix moves the auto_map extraction before the early-exit check and adds tokenizer_auto_map is None to the condition, so models with custom tokenizers properly use the dynamic module loading path. * style --------- Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> Co-authored-by: vasqu <antonprogamer@gmail.com>