llama.cpp
vocab : keep DNA k-mer ids distinct from colliding BPE tokens
#23466

Merged

vocab : keep DNA k-mer ids distinct from colliding BPE tokens #23466

CISC merged 2 commits into ggml-org:master from kashif:fix-hybriddna-kmer-collision

kashif requested a review from

CISC 45 days ago

kashif force pushed from 28dea284 to da621b9c 45 days ago

kashif force pushed from da621b9c to 3f4eab02 45 days ago

kashif force pushed from 3f4eab02 to 29e2bec5 45 days ago

kashif force pushed from 29e2bec5 to 745bfa26 45 days ago

vocab : mark hybriddna k-mers to avoid BPE token collisions

ed2329cf

kashif force pushed from 745bfa26 to ed2329cf 45 days ago

improved loop

6cbf9dc4

github-actions added python

CISC approved these changes on 2026-05-21

CISC added merge ready

ServeurpersoCom approved these changes on 2026-05-22

CISC merged afcda09d into master 44 days ago

kashif deleted the fix-hybriddna-kmer-collision branch 44 days ago

Reviewers

ServeurpersoCom

CISC

Assignees

No one assigned

Labels

python merge ready

Milestone

No milestone