vllm
93a126fb - [Misc] Make cached tokenizer pickle-compatible (#17048)

Commit

116 days ago

[Misc] Make cached tokenizer pickle-compatible (#17048) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

References

Author

DarkLight1337

Parents

benchmarks
- benchmark_prefix_caching.py
tests/tokenization
- test_cached_tokenizer.py
vllm/transformers_utils
- tokenizer.py
- tokenizer_base.py
- tokenizers
  - mistral.py