transformers
57b980a6 - Fix saving FlaubertTokenizer configs (#14991)
Commit
View On GitHub
Commit
4 years ago
Fix saving FlaubertTokenizer configs (#14991) All specific tokenizer config properties must be passed to its base class (XLMTokenizer) in order to be saved. This was not the case for do_lowercase config. Thus it was not saved by save_pretrained() method and saving and reloading the tokenizer changed its behaviour. This commit fixes it.
Author
vmaryasin vmaryasin
Parents
16f0b7d7
FAQ Terms Privacy Refunds Impressum
Loading