PR #281 Improve BPE - SemanticDiff

Improve BPE #281

apaniukov merged 12 commits into openvinotoolkit:master from pavel-esir:improve_bpe

pavel-esir added bug

pavel-esir requested a review from

apaniukov 1 year ago

github-actions added tokenizers_cpp

pavel-esir commented on 2024-10-10

pavel-esir force pushed from a7391e6f to 865da349 1 year ago

andrei-kochin added this to the 2024.5 milestone 1 year ago

pavel-esir force pushed from 865da349 to 7ed3e477 1 year ago

github-actions added tokenizers_test

andrei-kochin added Code Freeze

pavel-esir force pushed from 7ed3e477 to af15476b 1 year ago

github-actions removed tokenizers_test

use list

bf71bf66

use priority queue

61d644cd

some corrections

13e03f19

thread safe use of RE2

3e2f3b3a

optimizations and cleanup

fe1fb578

wip

c31cf611

fix segfault

62e0a9cd

use shared_ptrs instead of raw pointers

af15476b

minor corrections

59563f79

Merge remote-tracking branch 'upstream/master' into improve_bpe

f494bcb1

apaniukov approved these changes on 2024-10-21

fix comparing in PriorityQueue

0b5d65e7

minor improvements

b50031b1

apaniukov merged 17fc3c1c into master 1 year ago

pavel-esir deleted the improve_bpe branch 1 year ago

Reviewers

apaniukov

Assignees

No one assigned

Labels

bug tokenizers_cpp Code Freeze

Milestone

2024.5

openvino_tokenizers Improve BPE #281 Merged

Improve BPE #281

openvino_tokenizers
Improve BPE
#281

Merged