PR #172 Add truncate to high level api

Add truncate to high level api #172

SagsMug wants to merge 14 commits into abetlen:main from SagsMug:truncate

Add truncate strategy

8c525d89

Add function calls for truncate

188f937a

Allow 0 missing for truncate

5bc7013b

Added safety margin to tokenize when using truncate

2c7cf4b1

Correct types for truncate

2bc0ff43

SagsMug changed the title ~~Add truncate to llama-cpp-python~~ Add truncate to high level api 2 years ago

Tokenize size double strat

eba37092

Fix a tokenize bug

8d826799

Wrong type fix for tokenize

704995cf

Merge branch 'main' of https://github.com/abetlen/llama-cpp-python in…

63c3a714

Merge branch 'main' of https://github.com/abetlen/llama-cpp-python in…

7cc4fe4e

Merge branch 'truncate' of https://github.com/SagsMug/llama-cpp-pytho…

1755aacb

Merge branch 'main' of https://github.com/abetlen/llama-cpp-python in…

20121922

gjmulder added enhancement

Merge branch 'main' of https://github.com/abetlen/llama-cpp-python in…

75b647fa

Fixed failed merge

796a4f17

SagsMug force pushed from 796a4f17 to 65ebb446 2 years ago

SagsMug force pushed from 65ebb446 to 9d997fd6 2 years ago

SagsMug force pushed from 9d997fd6 to 796a4f17 2 years ago

abetlen force-pushed the main branch from 0af6e495 to 3337a980 1 year ago

abetlen force-pushed the main branch from 8c93cf86 to cc0fe438 1 year ago

Reviewers

No reviews

Assignees

No one assigned

Labels

enhancement

Milestone

No milestone