llama.cpp
vocab : keep DNA k-mer ids distinct from colliding BPE tokens
#23466

Merged

Commits

vocab : mark hybriddna k-mers to avoid BPE token collisions

kashif committed 107 days ago
improved loop

CISC committed 107 days ago

Loading