transformers
Updated documentation and added conversion utility
#34319

Merged

Updated documentation and added conversion utility #34319

ArthurZucker merged 7 commits into huggingface:main from viktor-shcherb:tiktoken_improved_documentation

stevhliu approved these changes on 2024-10-25

ArthurZucker commented on 2024-10-29

ArthurZucker commented on 2024-11-25

viktor-shcherb force pushed 1 year ago

Updated documentation and added conversion utility

3deb7640

Update docs/source/en/tiktoken.md

200c0208

Update docs/source/en/tiktoken.md

3ce18281

Moved util function to integration folder + allow for str

a9550576

Update formatting

326fcdce

viktor-shcherb force pushed to 326fcdce 1 year ago

Updated formatting

656d1617

style changes

afcddba2

ArthurZucker approved these changes on 2024-11-25

ArthurZucker merged 95c10fed into main 1 year ago

EndlessReform commented on 2024-12-07

Reviewers

ArthurZucker

stevhliu

EndlessReform

Assignees

No one assigned

Labels

None yet

Milestone

No milestone