transformers
eb4579cf - `tokenizer` train from iterator without pre_tokenizers (#35396)

Commit

358 days ago

`tokenizer` train from iterator without pre_tokenizers (#35396) * fix if else issues * add a test * fix the test * style

References

Author

ArthurZucker

Parents