PR #79 Fix UTF-8 handling (including colors)

Fix UTF-8 handling (including colors) #79

ggerganov merged 1 commit into ggml-org:master from kharvd:kharvd/tokenizer-fix

Fix UTF-8 handling (including colors)

cb8c464a

beiller approved these changes on 2023-03-13

ggerganov approved these changes on 2023-03-13

ggerganov merged 2a20f48e into master 2 years ago

Reviewers

ggerganov

beiller

Assignees

No one assigned

Labels

None yet

Milestone

No milestone