transformers
9956c2bc - Add a fix for custom code tokenizers in pipelines (#32300)

Commit
2 years ago
Add a fix for custom code tokenizers in pipelines (#32300) * Add a fix for the case when tokenizers are passed as a string * Support image processors and feature extractors as well * Reverting load_feature_extractor and load_image_processor * Add test * Test is torch-only * Add tests for preprocessors and feature extractors and move test * Extremely experimental fix * Revert that change, wrong branch! * Typo! * Split tests
Author
Parents
Loading