bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits

Muennighoff committed 3 years ago

86c74e68

MTF dataset and packing (#293)

thomasw21 committed 3 years ago

Verified c5b88fb9

Merge MLM too fast 2 (#294)

thomasw21 committed 3 years ago

Verified 131bd43e

Eval harness (#212)

DanielHesslow committed 3 years ago

Verified 3ab0ad18

Fixed MLM dataset arguments(#290)

thomasw21 committed 3 years ago

Verified 55f8cf8b

Mlm adaptation (#287)

Lintang Sutawika committed 3 years ago

Verified 9d264312

Explain many-to-many status

Muennighoff committed 3 years ago

e8ba66af

Add many-to-many non-ZeRO reshaping

Muennighoff committed 3 years ago

c14df232

Merge branch 'main' into nozero_reshape

Muennighoff committed 3 years ago

d94e1e24

Fix DS init (#285)

Quentin-Anthony committed 3 years ago

Verified 987663c1

tjruwase committed 3 years ago

a247614b

Parallel slice merging

tjruwase committed 3 years ago

e8a1ccf1

stas00 committed 3 years ago

721380b2

stas00 committed 3 years ago

bca5af4e

close+join pool; add tqdm; comment out noise

stas00 committed 3 years ago

04d9ad0f

Parallelize shard extraction

tjruwase committed 3 years ago

cc8810be

Fix tflops glu computation (#283)

Muennighoff committed 3 years ago

Verified e23393fb

args should be required, don't create another latest file

stas00 committed 3 years ago

795fedbb

Debug functions

tjruwase committed 3 years ago

55bb5148

tjruwase committed 3 years ago

7ae002d3

tjruwase committed 3 years ago

3bf14fdf

tjruwase committed 3 years ago

dd0aeb67

tjruwase committed 3 years ago

f29bacc1

make the tests adaptable to the number of available gpus

stas00 committed 3 years ago

f01fa4a5

[valid] deadlock workaround (#282)

stas00 committed 3 years ago

Verified cb48bd2c

tjruwase committed 3 years ago

85ff56ca

all gpu1<->2 reshapes work

stas00 committed 3 years ago

d5e33dec

Loading universal checkpoint with reshaping

tjruwase committed 3 years ago

ab0a7f8f

stas00 committed 3 years ago

d3005120

wip on fragments dealing

stas00 committed 3 years ago

7e0a81b9

Older