openvino_tokenizers
[GGUF] Create tokenizers factory for GGUF support in OpenVINO GenAI
#494

Merged

[GGUF] Create tokenizers factory for GGUF support in OpenVINO GenAI #494

rkazants merged 4 commits into openvinotoolkit:releases/2025/2 from rkazants:create_tokenizers_factory_2025_2

[GGUF] Create tokenizers factory method

2c15530d

Clean-up implementation

82fd33ae

github-actions added tokenizers_cpp

rkazants changed the title ~~Create tokenizers factory 2025 2~~ [GGUF ]Create tokenizers factory for GGUF support in OpenVINO GenAI 258 days ago

rkazants changed the title ~~[GGUF ]Create tokenizers factory for GGUF support in OpenVINO GenAI~~ [GGUF] Create tokenizers factory for GGUF support in OpenVINO GenAI 258 days ago

rkazants commented on 2025-05-26

Update src/tokenizers_factory.cpp

67ab09b0

rkazants commented on 2025-05-26

Update src/tokenizers_factory.cpp

2aca44d7

rkazants requested a review from

pavel-esir 258 days ago

rkazants requested a review from

gkrivor 258 days ago

rkazants requested a review from

andrei-kochin 258 days ago

rkazants enabled auto-merge (squash) 258 days ago

pavel-esir approved these changes on 2025-05-26

rkazants merged 85be884a into releases/2025/2 258 days ago

Reviewers

pavel-esir

gkrivor

andrei-kochin

Assignees

No one assigned

Labels

tokenizers_cpp

Milestone

No milestone

openvino_tokenizers [GGUF] Create tokenizers factory for GGUF support in OpenVINO GenAI #494 Merged

[GGUF] Create tokenizers factory for GGUF support in OpenVINO GenAI #494

openvino_tokenizers
[GGUF] Create tokenizers factory for GGUF support in OpenVINO GenAI
#494

Merged