transformers
7418a48e - Removed `Bert` interdependency in `tokenization_electra.py` (#19356)

Commit
3 years ago
Removed `Bert` interdependency in `tokenization_electra.py` (#19356) * Copied from BertTokenizer() in tokenization_bert * Added BasicTokenizer and WordPieceTokenizer Class * Update src/transformers/models/electra/tokenization_electra.py Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Added copied from comments for basicTokenizer and WordPieceTokenizer * Updated the comments for the tokenizerClasses * Update src/transformers/models/electra/tokenization_electra.py Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Update src/transformers/models/electra/tokenization_electra.py Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Formatted tokenization_electra with `make style` * Fix repo inconsistencies * Update src/transformers/models/electra/tokenization_electra.py Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Set the logger Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Author
Parents
Loading