transformers

716df5fb - [ `TokenizationUtils`] Fix `add_special_tokens` when the token is already there (#28520)

[ `TokenizationUtils`] Fix `add_special_tokens` when the token is already there (#28520) * fix adding special tokens when the token is already there. * add a test * add a test * nit * fix the test: make sure the order is preserved * Update tests/test_tokenization_common.py Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com> --------- Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>