bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits
make the tests adaptable to the number of available gpus
stas00 committed 3 years ago
f01fa4a5
[valid] deadlock workaround (#282)
stas00 committed 3 years ago
Verified cb48bd2c
param attrs
tjruwase committed 3 years ago
85ff56ca
all gpu1<->2 reshapes work
stas00 committed 3 years ago
d5e33dec
Loading universal checkpoint with reshaping
tjruwase committed 3 years ago
ab0a7f8f
cleanup
stas00 committed 3 years ago
d3005120
wip on fragments dealing
stas00 committed 3 years ago
7e0a81b9
rename
stas00 committed 3 years ago
9c447933
rename
stas00 committed 3 years ago
c29d3369
wip universal chkpt
stas00 committed 3 years ago
804b497d
convert to bf16
stas00 committed 3 years ago
6d863582
Merge remote-tracking branch 'origin/main' into ds_ckpt_reshape
stas00 committed 3 years ago
a3ef7783
Fix mixed fused layer norm to mimick nn.LayerNorm for torch>1.11 (#281)
thomasw21 committed 3 years ago
Verified 908dc9cb
Update CODEOWNERS
TevenLeScao committed 3 years ago
Verified c85b7c25
Update CODEOWNERS
TevenLeScao committed 3 years ago
Verified b6266f55
Update CODEOWNERS
TevenLeScao committed 3 years ago
Verified c6f22c41
Create CODEOWNERS
TevenLeScao committed 3 years ago
Verified 89c343f7
tweak the doc
stas00 committed 3 years ago
4c13c617
add start-fast doc (#278)
stas00 committed 3 years ago
Verified 40bd933d
add stop alarm instructions
stas00 committed 4 years ago
Verified 640c8180
`torch.testing.assert_equal` didn't make it (#273)
stas00 committed 4 years ago
Verified 87a9dba0
[tensorboard] add rename and remove event tools (#269)
stas00 committed 4 years ago
Verified affff3d2
[kill switch] fix test (#268)
stas00 committed 4 years ago
Verified 26feccc1
disable samples-per-dataset, steps-per-dataset, tokens-per-dataset (#267)
stas00 committed 4 years ago
Verified de3a0277
[kill switch] correct sys.exit (#266)
stas00 committed 4 years ago
Verified 1893811e
Sorry, last change was meant to a PR. This reverts commit d0fcf4170def7205426117016d4622c745f33883.
TevenLeScao committed 4 years ago
497aa1bf
preprocessing from arrow file to load an HF dataset
TevenLeScao committed 4 years ago
d0fcf417
switch to MixedFusedLayerNorm (#262)
stas00 committed 4 years ago
Verified b9883f44
allocate embed norm only on pp0 (#261)
stas00 committed 4 years ago
Verified dd06ea32
sync the whole Meg-LM fused_kernels sub-tree (#260)
stas00 committed 4 years ago
Verified 1cb76a64
Terms Of Use Privacy Policy Refund Policy Impressum