transformers.js
c91248cf - Add support for RoFormer models (#464)

Commit

1 year ago

Add support for RoFormer models (#464) * Add `RoFormerTokenizer * Use `clean_text` in bert normalizer config * Add control characters test * Add support for RoFormer models * Use default label if id2label is not specified * Update requirements.txt * Skip roformer tokenizer tests

References

#464 - Add support for RoFormer models

Author

xenova

Parents

7636a1c4

Files8

README.md
docs/snippets
- 6_supported-models.snippet
scripts
- supported_models.py
src
- models.js
- pipelines.js
- tokenizers.js
tests
- generate_tests.py
- requirements.txt

transformers.js c91248cf - Add support for RoFormer models (#464)

transformers.js
c91248cf - Add support for RoFormer models (#464)