bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits
ckpt averaging
Muennighoff committed 3 years ago
a36f2244
not yet working script
stas00 committed 3 years ago
7b5f175b
Create README.md
stas00 committed 3 years ago
Verified 2ce8bb4f
Fix causal attention mask (#306)
thomasw21 committed 3 years ago
Verified 38607ae9
Add bias a weight we need to sync as well (#307)
thomasw21 committed 3 years ago
Verified 0d0d84c8
Combine Specs (#304)
Muennighoff committed 3 years ago
Verified c3be5d3f
Add support for weighted train (#299)
thomasw21 committed 3 years ago
Verified 43ab0e08
MTF train script (#295)
thomasw21 committed 3 years ago
Verified 3d5d1514
sync layer norms (#272)
stas00 committed 3 years ago
Verified e1c479e5
CI fixes (#302)
stas00 committed 3 years ago
Verified 0cb043cf
MTF dataset and packing (#293)
thomasw21 committed 3 years ago
Verified c5b88fb9
Merge MLM too fast 2 (#294)
thomasw21 committed 3 years ago
Verified 131bd43e
Eval harness (#212)
DanielHesslow committed 3 years ago
Verified 3ab0ad18
Fixed MLM dataset arguments(#290)
thomasw21 committed 3 years ago
Verified 55f8cf8b
Mlm adaptation (#287)
Lintang Sutawika committed 3 years ago
Verified 9d264312
Fix DS init (#285)
Quentin-Anthony committed 3 years ago
Verified 987663c1
Fix tflops glu computation (#283)
Muennighoff committed 3 years ago
Verified e23393fb
[valid] deadlock workaround (#282)
stas00 committed 3 years ago
Verified cb48bd2c
Fix mixed fused layer norm to mimick nn.LayerNorm for torch>1.11 (#281)
thomasw21 committed 3 years ago
Verified 908dc9cb
Update CODEOWNERS
TevenLeScao committed 3 years ago
Verified c85b7c25
Update CODEOWNERS
TevenLeScao committed 3 years ago
Verified b6266f55
Update CODEOWNERS
TevenLeScao committed 3 years ago
Verified c6f22c41
Create CODEOWNERS
TevenLeScao committed 3 years ago
Verified 89c343f7
tweak the doc
stas00 committed 3 years ago
4c13c617
add start-fast doc (#278)
stas00 committed 3 years ago
Verified 40bd933d
add stop alarm instructions
stas00 committed 3 years ago
Verified 640c8180
`torch.testing.assert_equal` didn't make it (#273)
stas00 committed 3 years ago
Verified 87a9dba0
[tensorboard] add rename and remove event tools (#269)
stas00 committed 3 years ago
Verified affff3d2
[kill switch] fix test (#268)
stas00 committed 3 years ago
Verified 26feccc1
disable samples-per-dataset, steps-per-dataset, tokens-per-dataset (#267)
stas00 committed 3 years ago
Verified de3a0277
Terms Of Use Privacy Policy Refund Policy Impressum