transformers
8d9c3836 - Add clean_up_tokenization_spaces to config (#22341)
Commit
View On GitHub
Commit
2 years ago
Add clean_up_tokenization_spaces to config (#22341) * add draft changes * fix failing wav2vec * style * make sure that the argument is saved + add tests * style * fixup * update test * default clean_up_tokenization_spaces to False for Bloom and Llama * Update code based on review Co-authored-by: Nicolas Patry <patry.nicolas@gmail.com> * style * quality --------- Co-authored-by: Nicolas Patry <patry.nicolas@gmail.com>
Author
ArthurZucker ArthurZucker
Parents
b29fd697
FAQ Terms Privacy Refunds Impressum
Loading