bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits
Updates
Muennighoff committed 3 years ago
03b497bc
Use prefix arg
Muennighoff committed 3 years ago
1fcd4133
Add PrefixLM eval
Muennighoff committed 3 years ago
d58d6437
Merge branch 'bseval_harness' of https://github.com/bigscience-workshop/Megatron-DeepSpeed into bseval_harness
Muennighoff committed 3 years ago
02961ea3
Add small model scripts
Muennighoff committed 3 years ago
9a2277ce
Merge branch 'main' into bseval_harness
Muennighoff committed 3 years ago
b4660096
not yet working script
stas00 committed 3 years ago
7b5f175b
Create README.md
stas00 committed 3 years ago
Verified 2ce8bb4f
Fix causal attention mask (#306)
thomasw21 committed 3 years ago
Verified 38607ae9
Add bias a weight we need to sync as well (#307)
thomasw21 committed 3 years ago
Verified 0d0d84c8
Remove codecarbon
thomasw21 committed 3 years ago
280f1dcd
There's no BOS for bloom
thomasw21 committed 3 years ago
43936d9f
Combine Specs (#304)
Muennighoff committed 3 years ago
Verified c3be5d3f
Filter out generative tasks
thomasw21 committed 3 years ago
6105fe4a
This is already done correctly
thomasw21 committed 3 years ago
c6f76025
Woops
thomasw21 committed 3 years ago
bd31b62c
Be very careful of random states
thomasw21 committed 3 years ago
79cb5693
Improvement
thomasw21 committed 3 years ago
c193ffc3
Add torch barrier
thomasw21 committed 3 years ago
01dc62ac
Add torch barrier
thomasw21 committed 3 years ago
d0b2efaa
Fix setup_example_logger
thomasw21 committed 3 years ago
93134669
Add support for weighted train (#299)
thomasw21 committed 3 years ago
Verified 43ab0e08
MTF train script (#295)
thomasw21 committed 3 years ago
Verified 3d5d1514
add offload arg to slurm scripts
Muennighoff committed 3 years ago
40cf8690
Add offload arg
Muennighoff committed 3 years ago
9cf7ffd9
Offload to CPU earlier & increase number of bs in pipleine parallelism
Muennighoff committed 3 years ago
f75af1f9
Add downloading
Muennighoff committed 3 years ago
9af3e02e
sync layer norms (#272)
stas00 committed 3 years ago
Verified e1c479e5
CI fixes (#302)
stas00 committed 3 years ago
Verified 0cb043cf
MTF dataset and packing (#293)
thomasw21 committed 3 years ago
Verified c5b88fb9
Terms Of Use Privacy Policy Refund Policy Impressum