Megatron-DeepSpeed
Enable loading ckpt for t0 finetuning
#309
Open
Overview
Commits
16
Changes
View On GitHub

Enable loading ckpt for t0 finetuning #309

Muennighoff wants to merge 16 commits into main from t0loading
Muennighoff
Muennighoff Enable loading ckpt for t0 finetuning
90b8f46d
thomasw21
thomasw21 commented on 2022-07-11
Muennighoff Swap decoder_is_inputs & segment_ids
abdd7030
Muennighoff Add prepend-space arg
0fcb19c1
thomasw21
thomasw21 commented on 2022-07-11
Muennighoff Update tools/preprocess_data.py
63daa46f
Muennighoff Add helpers & set is_causal to true
89460c0a
Muennighoff Merge remote
fb8ecb8c
Muennighoff JSON helper scripts
a55d2fb5
Muennighoff Remove unnec imports
2dfe5d11
Muennighoff Remove helper scripts
ca740f1e
thomasw21
thomasw21 commented on 2022-07-12
Muennighoff Avoid loading module when not loading optim
cb0313ba
Muennighoff Allow not using torch distributed
b62dcafc
Muennighoff Add prefixlm arg
b15ca2d5
Muennighoff Muennighoff requested a review from thomasw21 thomasw21 3 years ago
Muennighoff Add bos option
dc8d0abb
Muennighoff Merge branch 'main' into t0loading
0a324592
Muennighoff Add reset-progress key
26997216
Muennighoff Add option to normalize loss per target (#326)
1e77844c
thomasw21
thomasw21 requested changes on 2022-11-04

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum