bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits
Add option to normalize loss per target (#326)
Muennighoff committed 3 years ago
Verified 1e77844c
Add reset-progress key
Muennighoff committed 3 years ago
26997216
Merge branch 'main' into t0loading
Muennighoff committed 3 years ago
0a324592
BLOOM Inference via DeepSpeed-Inference, Accelerate and DeepSpeed-ZeRO (#308)
stas00 committed 3 years ago
Verified 3932c749
Add bos option
Muennighoff committed 3 years ago
dc8d0abb
Reshape deepspeed checkpoint (#239)
tjruwase committed 3 years ago
Verified 0f23a729
Add prefixlm arg
Muennighoff committed 3 years ago
b15ca2d5
Allow not using torch distributed
Muennighoff committed 3 years ago
b62dcafc
Avoid loading module when not loading optim
Muennighoff committed 3 years ago
cb0313ba
Remove helper scripts
Muennighoff committed 3 years ago
ca740f1e
Remove unnec imports
Muennighoff committed 3 years ago
2dfe5d11
JSON helper scripts
Muennighoff committed 3 years ago
a55d2fb5
Merge remote
Muennighoff committed 3 years ago
fb8ecb8c
Add helpers & set is_causal to true
Muennighoff committed 3 years ago
89460c0a
Update tools/preprocess_data.py
Muennighoff committed 3 years ago
Verified 63daa46f
Add prepend-space arg
Muennighoff committed 3 years ago
0fcb19c1
Swap decoder_is_inputs & segment_ids
Muennighoff committed 3 years ago
abdd7030
Enable loading ckpt for t0 finetuning
Muennighoff committed 3 years ago
90b8f46d
not yet working script
stas00 committed 3 years ago
7b5f175b
Create README.md
stas00 committed 3 years ago
Verified 2ce8bb4f
Fix causal attention mask (#306)
thomasw21 committed 3 years ago
Verified 38607ae9
Add bias a weight we need to sync as well (#307)
thomasw21 committed 3 years ago
Verified 0d0d84c8
Combine Specs (#304)
Muennighoff committed 3 years ago
Verified c3be5d3f
Add support for weighted train (#299)
thomasw21 committed 3 years ago
Verified 43ab0e08
MTF train script (#295)
thomasw21 committed 3 years ago
Verified 3d5d1514
sync layer norms (#272)
stas00 committed 3 years ago
Verified e1c479e5
CI fixes (#302)
stas00 committed 3 years ago
Verified 0cb043cf
MTF dataset and packing (#293)
thomasw21 committed 3 years ago
Verified c5b88fb9
Merge MLM too fast 2 (#294)
thomasw21 committed 3 years ago
Verified 131bd43e
Eval harness (#212)
DanielHesslow committed 3 years ago
Verified 3ab0ad18
Terms Of Use Privacy Policy Refund Policy Impressum