Pull Requests bigscience-workshop/Megatron-DeepSpeed

[sync] bf16

#250 by stas00 was merged 2022-02-28 20:46

[apex FusedAdam] crash workaround

#249 by stas00 was merged 2022-02-18 02:10

[low CPU RAM] allocate on gpu directly, stagger checkpoint load/save

#248 by stas00 was closed 2022-03-07 18:49

implement --abort-on-unmet-fused-kernel-constraints

#247 by stas00 was merged 2022-02-08 02:34

Update pretrain_gpt3_175B.sh

#246 by Season111 was closed 2022-03-22 02:54

implement kill switch

#245 by stas00 was merged 2022-02-05 03:18

Alibi Tensor Parallel Fix

#244 by DanielHesslow was merged 2022-02-01 15:01

Add support for SentencePiece tokenisers

#243 by samsontmr was closed 2022-02-02 14:29

Fix throughput unit

#241 by janEbert was merged 2022-01-28 01:36

fix `add_scalar` for pt<1.9

#240 by stas00 was merged 2022-01-24 23:31

Reshape deepspeed checkpoint

#239 by tjruwase was merged 2022-07-20 20:45

Reshaping DeepSpeed checkpoints

#238 by tjruwase was closed 2022-01-24 22:48

enable new_style for add_scalar function for faster data format

#237 by abodacs was merged 2022-01-21 21:05

[TB] log restarts

#234 by stas00 was merged 2022-01-29 16:56

fix tests

#232 by stas00 was merged 2022-01-15 03:42

implement missing --no-load-optim support for deepspeed path

#231 by stas00 was merged 2022-01-15 02:55

Revert "tflops for CL"

#225 by stas00 was merged 2022-01-06 00:13

tflops for CL

#224 by stas00 was merged 2022-01-05 21:25

fix tflops calculation

#223 by stas00 was merged 2022-01-05 20:57

Fix alibi

#222 by thomasw21 was merged 2022-01-06 12:14

[WIP] fix alibi positional embeddings for training

#219 by SaulLu was closed 2022-01-04 16:20

save args to txt file

#218 by bhavitvyamalik was merged 2022-01-13 00:27

[logging] synced print

#217 by stas00 was merged 2021-12-16 02:54

[sync] replay layer_norm_cuda_kernel.cu fixes

#216 by stas00 was merged 2021-12-10 04:10

[ds report] less noise

#215 by stas00 was merged 2022-01-21 20:47

Eval harness

#212 by DanielHesslow was merged 2022-06-28 14:47

Fix consumed_valid_samples counting for several valid dataloaders

#211 by TevenLeScao was merged 2021-12-01 14:23

[TB] add throughput graphs

#210 by stas00 was merged 2021-12-07 02:21

Checking we use fused kernels to compute scaled masked softmax on prefix lm

#209 by thomasw21 was merged 2021-11-26 16:06

[CUDA extensions] improve build

#207 by stas00 was merged 2021-12-13 21:53