llama.cpp
4cd621c2 - convert : add BPE pre-tokenization for DBRX (#7132)

Commit

1 year ago

convert : add BPE pre-tokenization for DBRX (#7132) * Add BPE pre-tokenization for DBRX. * Add vocab GGUFs. * Remove test. * Remove GGUFs.

References

Author

dranger003

Parents