bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits
[sync] bf16 (#250)
stas00 committed 3 years ago
Verified 49d21af2
Try to read the data path arguments directly from a file (#254)
thomasw21 committed 3 years ago
Verified cf2d23a5
Use padded vocab size in preprocessing scripts (#253)
thomasw21 committed 3 years ago
Verified 99c8fe07
Fix preprocess_data_many_cores to use dtype
thomasw21 committed 3 years ago
b289eb81
Fix preprocess_data_many_cores to use dtype
thomasw21 committed 3 years ago
2c43a0fe
Replace approximate formula with exact one for throughput (#251)
deepakn94 committed 3 years ago
Verified ae6277f5
[apex FusedAdam] crash workaround (#249)
stas00 committed 3 years ago
Verified 541b967f
--abort-on-unmet-fused-kernel-constraints (#247)
stas00 committed 3 years ago
Verified 406022ed
implement kill switch (#245)
stas00 committed 3 years ago
Verified e9c8550e
Alibi Tensor Parallel Fix (#244)
DanielHesslow committed 3 years ago
Verified 40e8b2a0
[TB] log restarts (#234)
stas00 committed 3 years ago
Verified dfb5d687
Fix throughput unit (#241)
janEbert committed 3 years ago
Verified 4c2aea04
fix add_scalar for pt<1.9 (#240)
stas00 committed 3 years ago
Verified 5da3578a
enable new_style for add_scalar function for faster data format (#237)
abodacs committed 4 years ago
Verified e9750c42
[ds report] less noise (#215)
stas00 committed 4 years ago
Verified bbf311ee
fix tests (#232)
stas00 committed 4 years ago
Verified 04c461ed
implement missing --no-load-optim support for deepspeed path (#231)
stas00 committed 4 years ago
Verified efefd8a4
save args to txt file (#218)
bhavitvyamalik committed 4 years ago
Verified 14a03170
Fix alibi (#222)
thomasw21 committed 4 years ago
Verified 7ab5c05a
Revert "tflops for CL (#224)" (#225)
stas00 committed 4 years ago
Verified d231e117
tflops for CL (#224)
stas00 committed 4 years ago
Verified e4f2a08f
fix tflops calculation (#223)
stas00 committed 4 years ago
Verified d1143a7e
[logging] synced print (#217)
stas00 committed 4 years ago
Verified 4ed19efb
improve build (#207)
stas00 committed 4 years ago
Verified 10386f89
replay layer_norm_cuda_kernel.cu fixes (#216)
stas00 committed 4 years ago
Verified 16e411e2
[TB] add throughput graphs (#210)
stas00 committed 4 years ago
Verified e53fd737
Fix consumed_valid_samples counting for several valid dataloaders
TevenLeScao committed 4 years ago
Verified 06cfbec7
Revert "Checking we use fused kernels to compute scaled masked softmax on prefix lm (#209)"
thomasw21 committed 4 years ago
c9afebc6
Checking we use fused kernels to compute scaled masked softmax on prefix lm (#209)
thomasw21 committed 4 years ago
Verified b227590f
Reweighting strat for prefix lm (#190)
thomasw21 committed 4 years ago
Verified b3cf1755
Terms Of Use Privacy Policy Refund Policy Impressum