bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits
Fix DS init (#285)
Quentin-Anthony committed 3 years ago
Verified 987663c1
Fix tflops glu computation (#283)
Muennighoff committed 3 years ago
Verified e23393fb
[valid] deadlock workaround (#282)
stas00 committed 3 years ago
Verified cb48bd2c
Fix mixed fused layer norm to mimick nn.LayerNorm for torch>1.11 (#281)
thomasw21 committed 3 years ago
Verified 908dc9cb
Update CODEOWNERS
TevenLeScao committed 3 years ago
Verified c85b7c25
Update CODEOWNERS
TevenLeScao committed 3 years ago
Verified b6266f55
Update CODEOWNERS
TevenLeScao committed 3 years ago
Verified c6f22c41
Create CODEOWNERS
TevenLeScao committed 3 years ago
Verified 89c343f7
tweak the doc
stas00 committed 3 years ago
4c13c617
add start-fast doc (#278)
stas00 committed 3 years ago
Verified 40bd933d
add stop alarm instructions
stas00 committed 3 years ago
Verified 640c8180
`torch.testing.assert_equal` didn't make it (#273)
stas00 committed 3 years ago
Verified 87a9dba0
[tensorboard] add rename and remove event tools (#269)
stas00 committed 3 years ago
Verified affff3d2
[kill switch] fix test (#268)
stas00 committed 3 years ago
Verified 26feccc1
disable samples-per-dataset, steps-per-dataset, tokens-per-dataset (#267)
stas00 committed 3 years ago
Verified de3a0277
[kill switch] correct sys.exit (#266)
stas00 committed 3 years ago
Verified 1893811e
Sorry, last change was meant to a PR. This reverts commit d0fcf4170def7205426117016d4622c745f33883.
TevenLeScao committed 3 years ago
497aa1bf
preprocessing from arrow file to load an HF dataset
TevenLeScao committed 3 years ago
d0fcf417
switch to MixedFusedLayerNorm (#262)
stas00 committed 3 years ago
Verified b9883f44
allocate embed norm only on pp0 (#261)
stas00 committed 3 years ago
Verified dd06ea32
sync the whole Meg-LM fused_kernels sub-tree (#260)
stas00 committed 3 years ago
Verified 1cb76a64
deploy elastic error handler (#258)
stas00 committed 3 years ago
Verified 8673d464
add `pad-vocab-size-to` argument and tests (#255)
SaulLu committed 3 years ago
Verified 58d92042
make partition_method configurable (#256)
stas00 committed 3 years ago
Verified 00f5f889
[sync] bf16 (#250)
stas00 committed 3 years ago
Verified 49d21af2
Try to read the data path arguments directly from a file (#254)
thomasw21 committed 3 years ago
Verified cf2d23a5
Use padded vocab size in preprocessing scripts (#253)
thomasw21 committed 3 years ago
Verified 99c8fe07
Fix preprocess_data_many_cores to use dtype
thomasw21 committed 3 years ago
b289eb81
Fix preprocess_data_many_cores to use dtype
thomasw21 committed 3 years ago
2c43a0fe
Replace approximate formula with exact one for throughput (#251)
deepakn94 committed 3 years ago
Verified ae6277f5
Terms Of Use Privacy Policy Refund Policy Impressum