transformers.js
4a31e549 - Add gemma tokenizer tests

Commit

240 days ago

Add gemma tokenizer tests

References

#545 - 🚀🚀🚀 Transformers.js V3 🚀🚀🚀

Author

xenova

xenova

Parents

Files2

tests/models
- all_tokenization_tests.js
- gemma
  - tokenization.js