PR #5992 llama : refactor unicode stuff

llama : refactor unicode stuff #5992

ggerganov merged 11 commits into master from gg/refactor-unicode

llama : refactor unicode stuff

0458996e

ggerganov marked this pull request as draft 1 year ago

unicode : names

9654d62f

make : fix c++ compiler

9f3f7d80

unicode : names

de0929ae

unicode : straighten tables

e607540e

zig : fix build

be12d8b1

unicode : put nfd normalization behind API

6568c62b

swift : fix build

4600538b

unicode : add BOM

58d54915

ggerganov marked this pull request as ready for review 1 year ago

unicode : add <cstdint>

af0621e6

slaren commented on 2024-03-11

unicode : pass as cpts as const ref

3680bc24

ggerganov force pushed to 3680bc24 1 year ago

ggerganov merged 83796e62 into master 1 year ago

ggerganov deleted the gg/refactor-unicode branch 1 year ago

Reviewers

slaren

Assignees

No one assigned

Labels

None yet

Milestone

No milestone