PR #2938 convert.py: BPE fixes?

convert.py: BPE fixes? #2938

ggerganov merged 2 commits into ggml-org:master from KerfuffleV2:fix-convert-bpe

convert.py: BPE fixes?

25c0d79b

Remove unnecessary conditional in addl token error handling

822127e6

ggerganov approved these changes on 2023-09-01

ggerganov merged cff7b0bf into master 2 years ago

KerfuffleV2 deleted the fix-convert-bpe branch 2 years ago

Reviewers

ggerganov

Assignees

No one assigned

Labels

None yet

Milestone

No milestone