bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits

Adding logging of data used per training dataset + 1 bugfix

TevenLeScao committed 4 years ago

cd20256b

rename extra-validation to periodic-eval

hadyelsahar committed 4 years ago

f7714076

add run script example for multilingual validation

hadyelsahar committed 4 years ago

64476947

hadyelsahar committed 4 years ago

20c48380

hadyelsahar committed 4 years ago

5a64b937

Extra_valid_dataset arg & display on Tensorboard

hadyelsahar committed 4 years ago

287f1247

adding multiple datasets to args

hadyelsahar committed 4 years ago

073c98ef

[CI] setting up a CI with EC2 backend (#78)

stas00 committed 4 years ago

Verified 9e14c02a

add missing dependencies (#88)

stas00 committed 4 years ago

Verified b4cc77fe

dynamically use as many 3d dimensions as possible (#83)

stas00 committed 4 years ago

Verified a025c8a1

Add step to download dataset before running the preprocess_data_dist test (#81)

adammoody committed 4 years ago

Verified fa9f54c9

fix test; skip broken test (#79)

stas00 committed 4 years ago

Verified 8427ca8f

distributed merge of per-rank Megatron data files (#55)

adammoody committed 4 years ago

Verified 97221116

stas00 committed 4 years ago

Verified 6d88ae20

[CI] start workflow (#75)

stas00 committed 4 years ago

Verified 3fd48db5

Check cardon directory is not None (#74)

thomasw21 committed 4 years ago

Verified e96df7dc

[codecarbon] integration (#15)

stas00 committed 4 years ago

Verified 350fb903

fix circular import (#72)

stas00 committed 4 years ago

Verified 4255845d

Expose GLU activations as arguments (#69)

jaketae committed 4 years ago

Verified b5a029d9

[testing] fixes for pt-1.10 (#71)

stas00 committed 4 years ago

Verified a96e2ab3

Add openwebtext1000.jsonl to .gitignore

thomasw21 committed 4 years ago

9341269d

- Recompute bin/idx using microsoft/Megatron-DeepSpeed (Not changes)

thomasw21 committed 4 years ago

5eeae0b8

fix: exclusive scan computing pointers list (#68)

adammoody committed 4 years ago

Verified 191a96b0

shuffle index list with numpy, scatter list, use file for large lists (#63)

adammoody committed 4 years ago

Verified a59d93c0

fix: doc_idx offset when merging indexed dataset files (#66)

adammoody committed 4 years ago

Verified c680954e

stas00 committed 4 years ago

1c1423ed

add testing and contribute info

stas00 committed 4 years ago

0d404320

fix arg help (#65)

stas00 committed 4 years ago

Verified 56c2983b

Add test suite (#64)

stas00 committed 4 years ago

Verified 55b7036c

use HuggingFace Datasets as source to build Megatron data files (#48)

adammoody committed 4 years ago

Verified 5069622a

Older