bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits
Merge remote-tracking branch 'origin/main' into eval_harness
stas00 committed 3 years ago
a7222590
tweak the doc
stas00 committed 3 years ago
4c13c617
add start-fast doc (#278)
stas00 committed 3 years ago
Verified 40bd933d
add stop alarm instructions
stas00 committed 3 years ago
Verified 640c8180
`torch.testing.assert_equal` didn't make it (#273)
stas00 committed 3 years ago
Verified 87a9dba0
Merge branch 'main' into eval_harness
stas00 committed 3 years ago
Verified a9221acc
[tensorboard] add rename and remove event tools (#269)
stas00 committed 3 years ago
Verified affff3d2
[kill switch] fix test (#268)
stas00 committed 3 years ago
Verified 26feccc1
disable samples-per-dataset, steps-per-dataset, tokens-per-dataset (#267)
stas00 committed 3 years ago
Verified de3a0277
[kill switch] correct sys.exit (#266)
stas00 committed 3 years ago
Verified 1893811e
Sorry, last change was meant to a PR. This reverts commit d0fcf4170def7205426117016d4622c745f33883.
TevenLeScao committed 3 years ago
497aa1bf
preprocessing from arrow file to load an HF dataset
TevenLeScao committed 3 years ago
d0fcf417
switch to MixedFusedLayerNorm (#262)
stas00 committed 3 years ago
Verified b9883f44
allocate embed norm only on pp0 (#261)
stas00 committed 3 years ago
Verified dd06ea32
sync the whole Meg-LM fused_kernels sub-tree (#260)
stas00 committed 3 years ago
Verified 1cb76a64
deploy elastic error handler (#258)
stas00 committed 3 years ago
Verified 8673d464
add `pad-vocab-size-to` argument and tests (#255)
SaulLu committed 3 years ago
Verified 58d92042
make partition_method configurable (#256)
stas00 committed 3 years ago
Verified 00f5f889
[sync] bf16 (#250)
stas00 committed 3 years ago
Verified 49d21af2
Try to read the data path arguments directly from a file (#254)
thomasw21 committed 3 years ago
Verified cf2d23a5
Use padded vocab size in preprocessing scripts (#253)
thomasw21 committed 3 years ago
Verified 99c8fe07
Fix preprocess_data_many_cores to use dtype
thomasw21 committed 3 years ago
b289eb81
Fix preprocess_data_many_cores to use dtype
thomasw21 committed 3 years ago
2c43a0fe
Replace approximate formula with exact one for throughput (#251)
deepakn94 committed 3 years ago
Verified ae6277f5
Update examples/run_evalharness_deepspeed.md
stas00 committed 3 years ago
Verified 1e06f41e
[apex FusedAdam] crash workaround (#249)
stas00 committed 3 years ago
Verified 541b967f
--abort-on-unmet-fused-kernel-constraints (#247)
stas00 committed 3 years ago
Verified 406022ed
implement kill switch (#245)
stas00 committed 3 years ago
Verified e9c8550e
Alibi Tensor Parallel Fix (#244)
DanielHesslow committed 3 years ago
Verified 40e8b2a0
[TB] log restarts (#234)
stas00 committed 3 years ago
Verified dfb5d687
Terms Of Use Privacy Policy Refund Policy Impressum