bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits

Add slurm script for p3 preparation

Muennighoff committed 3 years ago

dc46613a

Add P3 preparation script

Muennighoff committed 3 years ago

44471018

Merge MLM too fast 2 (#294)

thomasw21 committed 3 years ago

Verified 131bd43e

Eval harness (#212)

DanielHesslow committed 3 years ago

Verified 3ab0ad18

Fixed MLM dataset arguments(#290)

thomasw21 committed 3 years ago

Verified 55f8cf8b

Mlm adaptation (#287)

Lintang Sutawika committed 3 years ago

Verified 9d264312

Fix DS init (#285)

Quentin-Anthony committed 3 years ago

Verified 987663c1

Fix tflops glu computation (#283)

Muennighoff committed 3 years ago

Verified e23393fb

[valid] deadlock workaround (#282)

stas00 committed 3 years ago

Verified cb48bd2c

Fix mixed fused layer norm to mimick nn.LayerNorm for torch>1.11 (#281)

thomasw21 committed 3 years ago

Verified 908dc9cb

Update CODEOWNERS

TevenLeScao committed 3 years ago

Verified c85b7c25

Update CODEOWNERS

TevenLeScao committed 3 years ago

Verified b6266f55

Update CODEOWNERS

TevenLeScao committed 3 years ago

Verified c6f22c41

Create CODEOWNERS

TevenLeScao committed 3 years ago

Verified 89c343f7

stas00 committed 3 years ago

4c13c617

add start-fast doc (#278)

stas00 committed 3 years ago

Verified 40bd933d

add stop alarm instructions

stas00 committed 3 years ago

Verified 640c8180

`torch.testing.assert_equal` didn't make it (#273)

stas00 committed 3 years ago

Verified 87a9dba0

[tensorboard] add rename and remove event tools (#269)

stas00 committed 3 years ago

Verified affff3d2

[kill switch] fix test (#268)

stas00 committed 3 years ago

Verified 26feccc1

disable samples-per-dataset, steps-per-dataset, tokens-per-dataset (#267)

stas00 committed 3 years ago

Verified de3a0277

[kill switch] correct sys.exit (#266)

stas00 committed 3 years ago

Verified 1893811e

Sorry, last change was meant to a PR. This reverts commit d0fcf4170def7205426117016d4622c745f33883.

TevenLeScao committed 3 years ago

497aa1bf

preprocessing from arrow file to load an HF dataset

TevenLeScao committed 3 years ago

d0fcf417

switch to MixedFusedLayerNorm (#262)

stas00 committed 3 years ago

Verified b9883f44

allocate embed norm only on pp0 (#261)

stas00 committed 3 years ago

Verified dd06ea32

sync the whole Meg-LM fused_kernels sub-tree (#260)

stas00 committed 3 years ago

Verified 1cb76a64

deploy elastic error handler (#258)

stas00 committed 3 years ago

Verified 8673d464

add `pad-vocab-size-to` argument and tests (#255)

SaulLu committed 3 years ago

Verified 58d92042

make partition_method configurable (#256)

stas00 committed 3 years ago

Verified 00f5f889

Older