llm-foundry
Making tokenizers optional in the building of LLMs
#1781
Merged
Overview
Commits
8
Changes
View On GitHub

Making tokenizers optional in the building of LLMs #1781

ethantang-db merged 8 commits into main from ethantang-db/tokenizers_optional
ethantang-db
ethantang-db making optimizers optional
d2bc2ae0
ethantang-db ethantang-db requested a review 1 year ago
ethantang-db ethantang-db requested a review 1 year ago
ethantang-db ethantang-db requested a review from dakinggg dakinggg 1 year ago
dakinggg
dakinggg commented on 2025-04-08
ethantang-db ethantang-db force pushed from 9e2f1558 to d2bc2ae0 1 year ago
ethantang-db lint
18610668
dakinggg
dakinggg approved these changes on 2025-04-08
ethantang-db more precommit hooks
535d747c
ethantang-db tokenizer_name can be none
2046e1b2
ethantang-db typo
f78937dc
ethantang-db fix test
668fffb7
ethantang-db lint
0d5cd992
ethantang-db specify tokenizer param
5f753dd7
ethantang-db ethantang-db merged 0c803a2d into main 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum