Pull Requests bigscience-workshop/Megatron-DeepSpeed

Merge MLM too fast 2

#294 by thomasw21 was merged 2022-06-30 15:02

MTF dataset and packing

#293 by thomasw21 was merged 2022-07-02 04:16

Merged MLM too fast.

#290 by thomasw21 was merged 2022-06-27 16:58

Mtf finetuning

#288 by lintangsutawika was closed 2022-07-07 14:17

Mlm adaptation

#287 by lintangsutawika was merged 2022-06-27 15:23

Initialize dist with DS

#285 by Quentin-Anthony was merged 2022-06-22 10:02

MLM adaptation and Multitask Finetuning

#284 by lintangsutawika was closed 2022-07-22 03:28

Fix tflops glu computation

#283 by Muennighoff was merged 2022-06-05 19:30

[valid] deadlock workaround

#282 by stas00 was merged 2022-05-31 23:38

Fix mixed fused layer norm to mimick nn.LayerNorm for torch>1.11

#281 by thomasw21 was merged 2022-05-03 08:31

[doc] start-fast instructions

#278 by stas00 was merged 2022-04-13 20:32

Thomas/olruwase/sync layer norms

#277 by thomasw21 was merged 2022-04-06 16:42

Make dataloader use another random generator

#276 by thomasw21 was merged 2022-04-06 17:10

`torch.testing.assert_equal` didn't make it

#273 by stas00 was merged 2022-03-25 02:12

sync layer norms

#272 by stas00 was merged 2022-07-04 23:51

[tensorboard] add rename and remove event tools

#269 by stas00 was merged 2022-03-18 03:23

[kill switch] fix test

#268 by stas00 was merged 2022-03-18 01:24

[TB] disable samples-per-dataset, steps-per-dataset, tokens-per-dataset

#267 by stas00 was merged 2022-03-18 00:49

[kill switch] correct sys.exit

#266 by stas00 was merged 2022-03-18 00:37

[embed norm] switch to apex MixedFusedLayerNorm

#262 by stas00 was merged 2022-03-09 17:00

allocate embed norm only on pp0

#261 by stas00 was merged 2022-03-07 04:24

sync the whole Meg-LM fused_kernels sub-tree

#260 by stas00 was merged 2022-03-07 02:47

Fix softmax

#259 by thomasw21 was closed 2022-03-07 09:22

deploy elastic error handler

#258 by stas00 was merged 2022-03-01 23:05

Fix padded vocab size on preprocessing scripts

#257 by thomasw21 was merged 2022-03-01 07:08

make partition_method configurable

#256 by stas00 was merged 2022-02-28 22:37

add `pad-vocab-size-to` argument and tests

#255 by SaulLu was merged 2022-03-01 10:22

Try to read the data path arguments directly from a file

#254 by thomasw21 was merged 2022-02-26 23:44

Fix preprocessing scripts

#253 by thomasw21 was merged 2022-02-25 14:56

Replace approximate formula with exact one for throughput

#251 by deepakn94 was merged 2022-02-22 02:09