transformers
781e4b13 - Adding `skip_special_tokens=True` to FillMaskPipeline (#9783)

Commit
5 years ago
Adding `skip_special_tokens=True` to FillMaskPipeline (#9783) * We most likely don't want special tokens in this output. * Adding `skip_special_tokens=True` to FillMaskPipeline - It's backward incompatible. - It makes for sense for pipelines to remove references to special_tokens (all of the other pipelines do that). - Keeping special tokens makes it hard for users to actually remove them because all models have different tokens (<s>, <cls>, [CLS], ....) * Fixing `token_str` in the same vein, and actually fix the tests too !
Author
Parents
Loading