Add a GPT-2 pretraining example (Reactant, HuggingFaceDatasets.jl) #2754
Add a GPT-2 pretraining example (Reactant, HuggingFaceDatasets.jl)
d4b9255f
Update the example's numbers for --precision bf16
dbfbce3b
CarloLucibello
marked this pull request as ready for review 32 days ago
Require HuggingFaceDatasets 0.4.3 in the GPT-2 example
c04a5465
Explain where the GPT-2 example's compile time goes
02241d5a
Simplify the GPT-2 example's CLI, token ids and Python glue
bbaa03cb
Cache the GPT-2 example's token blocks with datasets instead of .bin …
30e2ac64
Use HuggingFaceDatasets 0.4.4 and MLUtils' sample_logits in the GPT-2…
1f64b98e
Share the GPT-2 example's token cache between Julia and PyTorch again
401236fb
Merge remote-tracking branch 'origin/master' into cl/gpt2-example
faa16a53
Use Flux's integer-label cross-entropy and encode_batch_ids in the GP…
4a9bd018
Assignees
No one assigned
Login to write a write a comment.
Login via GitHub