openvino_tokenizers
298e1ee3 - Store tokenizer conversion params in rt_info / refactor passing params (#268)

Commit
1 year ago
Store tokenizer conversion params in rt_info / refactor passing params (#268) * WIP * refactor passing parameters as a single object, stored rt_info * ready for review * apply review comments * apply ruff format * fix what ruff has broken * fix docstring * fix tests * add tests * Update python/openvino_tokenizers/convert_tokenizer.py Co-authored-by: Artur Paniukov <chgk1101@gmail.com> * add barier for specifying both params and individual conversion arguments * simplify more * fix tests 2 * move imports * fix tests 3 --------- Co-authored-by: Artur Paniukov <chgk1101@gmail.com>
Author
Parents
Loading