openvino_tokenizers
port 'use PCRE2 fallback for RegexNormalization' to release
#203

Merged

port 'use PCRE2 fallback for RegexNormalization' to release #203

apaniukov merged 14 commits into openvinotoolkit:releases/2024/3 from pavel-esir:add_pcre2_to_release

add pcre2

293d52e8

use string with length instead of ZERO_TERMINATION

0ad52b39

add Licence to thirt party

29a46dce

set m_search_pattern_re to nullptr if PCRE2 is used

bf980e15

pavel-esir requested a review from

apaniukov 1 year ago

remove unnecessary const char*

8a24e663

github-actions added tokenizers_cpp

github-actions added tokenizers_py

github-actions added tokenizers_test

github-actions added build

fix typo in m_replace_pattern

6fd785ec

pavel-esir changed the title ~~Add pcre2 to release~~ port 'use PCRE2 fallback for RegexNormalization' to release 1 year ago

Update Del Control Chars Regex

11b635c1

Update Pass Rate

41b6a610

fix win build

7aebfba6

revert added_tests in conftest.py

fd880b8a

revert Readme & pass rates

4f908340

update README & pass_rates

a65e7cbc

do not log errors

4dfea421

add comment on malloc

a64c6983

apaniukov approved these changes on 2024-07-24

apaniukov merged fb0157c3 into releases/2024/3 1 year ago

pavel-esir deleted the add_pcre2_to_release branch 1 year ago

Reviewers

apaniukov

Assignees

No one assigned

Labels

tokenizers_cpp tokenizers_py tokenizers_test build

Milestone

No milestone

openvino_tokenizers port 'use PCRE2 fallback for RegexNormalization' to release #203 Merged

port 'use PCRE2 fallback for RegexNormalization' to release #203

openvino_tokenizers
port 'use PCRE2 fallback for RegexNormalization' to release
#203

Merged