transformers
0c98f248 - fix: add __bool__ operator to tokenizer to avoid bloated asserts (#38899)

Commit
225 days ago
fix: add __bool__ operator to tokenizer to avoid bloated asserts (#38899) * fix: add __bool__ operator to tokenizer to avoid bloated asserts When a user does 'assert tokenizer' to ensure that the tokenizer is not None, they inadvertently set off a rather expensive process in the '__len__()' operator. This fix adds a trivial '__bool__()' that returns True, so that a None tokenizer asserts and an actual tokenizer returns True when asserted, without calling length op. * typo
Author
Parents
Loading