PR #33344 Fix Megatron-LM tokenizer path

Fix Megatron-LM tokenizer path #33344

ArthurZucker merged 7 commits into huggingface:main from hikari-kubota-fixstars:update-megatron-lm-tokenizer-path

Change Megatron-LM tokenizer path

d9ddb8ca

ArthurZucker commented on 2024-09-06

Add version check

19bc3477

Fix code formatting issues

9368f11a

hikari-kubota-fixstars requested a review from

ArthurZucker 1 year ago

ArthurZucker commented on 2024-09-13

Check module importability using importlib.util

e97d76e5

Fix code formatting issues

25241c5b

hikari-kubota-fixstars requested a review from

ArthurZucker 1 year ago

ArthurZucker commented on 2024-09-21

Use packaging library

cd64c827

Trigger CircleCI

04f1ae39

hikari-kubota-fixstars requested a review from

ArthurZucker 1 year ago

ArthurZucker approved these changes on 2024-09-25

ArthurZucker merged 68049b17 into main 1 year ago

hikari-kubota-fixstars deleted the update-megatron-lm-tokenizer-path branch 1 year ago

Reviewers

ArthurZucker

Assignees

No one assigned

Labels

None yet

Milestone

No milestone