bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits

thomasw21 committed 3 years ago

606fdeb5

Get pretrain on non causal mlm script

thomasw21 committed 3 years ago

a45c9cd4

MTF optimize dataloading (#298)

thomasw21 committed 3 years ago

Verified f2df7715

Update tests/test_model.py

thomasw21 committed 3 years ago

Verified 16c556c0

Cuda kernel is not strictly equivalent

thomasw21 committed 3 years ago

e3e21f55

Fix empty samples

thomasw21 committed 3 years ago

de4934f5

thomasw21 committed 3 years ago

0450bad8

thomasw21 committed 3 years ago

b321a349

Get test working

thomasw21 committed 3 years ago

41e931a9

Revert back changes linked to shutil.copytree

thomasw21 committed 3 years ago

985cd028

thomasw21 committed 3 years ago

5c806992

Change the condition for empty slice

thomasw21 committed 3 years ago

bebb481a

Prevent empty dataset

thomasw21 committed 3 years ago

1572ddc9

Build the index from scratch everytime

thomasw21 committed 3 years ago

37ad57e6

Silently skip samples that are too long

thomasw21 committed 3 years ago

5816adfb

thomasw21 committed 3 years ago

ef20e57a

thomasw21 committed 3 years ago

b759a92a

thomasw21 committed 3 years ago

1c2284f1

thomasw21 committed 3 years ago

d7a6388a

thomasw21 committed 3 years ago

de88ab63

Merge remote-tracking branch 'official_repo/main' into thomas/mtf_train_script

thomasw21 committed 3 years ago

efeb55a1

thomasw21 committed 3 years ago

e1a37677

thomasw21 committed 3 years ago

f4a4733e

Find a way to reset states everytime

thomasw21 committed 3 years ago

74e03ec4

Find a way to reset states everytime

thomasw21 committed 3 years ago

2e469e5a

Find a way to reset states everytime

thomasw21 committed 3 years ago

48a55b9a

Find a way to reset states everytime

thomasw21 committed 3 years ago

4e0c4caf

Find a way to reset states everytime

thomasw21 committed 3 years ago

d9719b6d

Requires to remember how may epochs

thomasw21 committed 3 years ago

20be5b90

thomasw21 committed 3 years ago

f5410765

Older