transformers
f2c8fe8d - add `tags_dict` to `tokenizer_config` in fast version

Commit

4 years ago

add `tags_dict` to `tokenizer_config` in fast version

References

#33 - Add a test to check if the `tags_dict` is correctly saved into `tokenizer_config.json`

Author

SaulLu

SaulLu

Parents

Loading