transformers
8ad06b7c - using raw string for regex to search <extra_id> (#21162)

Commit

2 years ago

using raw string for regex to search <extra_id> (#21162) * using raw string for regex to search <extra_id> * fix the same issue in test file:`tokenization_t5.py`

References

#27720 - Add common processor tests

#32831 - [Docs] Update resources

#29969 - [SigLIP] Add fast tokenizer

#33111 - [Backbone] Remove out_features everywhere

#33174 - [Zero-shot image classification pipeline] Remove tokenizer_kwargs

#21162 - using raw string for regex to search <extra_id>

Author

pfliu-nlp

Parents

8a17da2f

Files2

src/transformers/models/t5
- tokenization_t5.py
tests/models/t5
- test_tokenization_t5.py

transformers 8ad06b7c - using raw string for regex to search <extra_id> (#21162)

transformers
8ad06b7c - using raw string for regex to search <extra_id> (#21162)