Flux.jl
Add a GPT-2 pretraining example (Reactant, HuggingFaceDatasets.jl)
#2754
Merged

Add a GPT-2 pretraining example (Reactant, HuggingFaceDatasets.jl) #2754

CarloLucibello merged 10 commits into master from cl/gpt2-example
CarloLucibello
CarloLucibello Add a GPT-2 pretraining example (Reactant, HuggingFaceDatasets.jl)
d4b9255f
CarloLucibello Update the example's numbers for --precision bf16
dbfbce3b
CarloLucibello CarloLucibello marked this pull request as ready for review 32 days ago
CarloLucibello Require HuggingFaceDatasets 0.4.3 in the GPT-2 example
c04a5465
CarloLucibello Explain where the GPT-2 example's compile time goes
02241d5a
CarloLucibello Simplify the GPT-2 example's CLI, token ids and Python glue
bbaa03cb
CarloLucibello Cache the GPT-2 example's token blocks with datasets instead of .bin …
30e2ac64
CarloLucibello Use HuggingFaceDatasets 0.4.4 and MLUtils' sample_logits in the GPT-2…
1f64b98e
CarloLucibello Share the GPT-2 example's token cache between Julia and PyTorch again
401236fb
CarloLucibello Merge remote-tracking branch 'origin/master' into cl/gpt2-example
faa16a53
CarloLucibello Use Flux's integer-label cross-entropy and encode_batch_ids in the GP…
4a9bd018
CarloLucibello CarloLucibello merged d004f141 into master 31 days ago
codecov

Login to write a write a comment.

Login via GitHub

Reviewers
No reviews
Assignees
No one assigned
Labels
Milestone