llama.cpp
1be97831 - fix: prevent segfault in tokenizer on highly repetitive input (#17786)

Commit
94 days ago
fix: prevent segfault in tokenizer on highly repetitive input (#17786) Add nosubs|optimize flags to std::regex constructors to prevent catastrophic backtracking when processing prompts with repeated identical characters (e.g., 'A' * 10000). The nosubs flag disables subgroup capture, significantly reducing memory usage and backtracking on uniform token sequences
Parents
Loading