bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits

[CI] start workflow (#75)

stas00 committed 4 years ago

Verified 3fd48db5

Check cardon directory is not None (#74)

thomasw21 committed 4 years ago

Verified e96df7dc

[codecarbon] integration (#15)

stas00 committed 4 years ago

Verified 350fb903

fix circular import (#72)

stas00 committed 4 years ago

Verified 4255845d

Expose GLU activations as arguments (#69)

jaketae committed 4 years ago

Verified b5a029d9

[testing] fixes for pt-1.10 (#71)

stas00 committed 4 years ago

Verified a96e2ab3

Add openwebtext1000.jsonl to .gitignore

thomasw21 committed 4 years ago

9341269d

- Recompute bin/idx using microsoft/Megatron-DeepSpeed (Not changes)

thomasw21 committed 4 years ago

5eeae0b8

fix: exclusive scan computing pointers list (#68)

adammoody committed 4 years ago

Verified 191a96b0

shuffle index list with numpy, scatter list, use file for large lists (#63)

adammoody committed 4 years ago

Verified a59d93c0

fix: doc_idx offset when merging indexed dataset files (#66)

adammoody committed 4 years ago

Verified c680954e

stas00 committed 4 years ago

1c1423ed

add testing and contribute info

stas00 committed 4 years ago

0d404320

fix arg help (#65)

stas00 committed 4 years ago

Verified 56c2983b

Add test suite (#64)

stas00 committed 4 years ago

Verified 55b7036c

use HuggingFace Datasets as source to build Megatron data files (#48)

adammoody committed 4 years ago

Verified 5069622a

[microsoft/Megatron-DeepSpeed sync] Commits including 2021-08-09 (#58)

stas00 committed 4 years ago

Verified 3c9d748b

Add GLU variants (#47)

jaketae committed 4 years ago

Verified effb2fb8

better comment on TB writer (`is_last_rank`)

VictorSanh committed 4 years ago

5e3963db

Fix pretrain_gpt_single_node example script to have only one occurence of lr-decay-style

thomasw21 committed 4 years ago

7228b89e

chore: add deepspeed as comment

jaketae committed 4 years ago

Verified f4566b85

Update README.md (#51)

lintangsutawika committed 4 years ago

Verified a6cbb3bb

Add LRU cache, add faster tokenization (#37)

huu4ontocord committed 4 years ago

Verified 36284576

Update requirements.txt (#46)

jaketae committed 4 years ago

Verified f919d0cf

rm `(s)` that slipped through

VictorSanh committed 4 years ago

49b9d7de

Group tensorboard metrics (#39)

VictorSanh committed 4 years ago

Verified 9e75429d

chore: update gitignore (#45)

jaketae committed 4 years ago

Verified 752e958c

fix attn_mask (#50)

stas00 committed 4 years ago

Verified 42fe3b39

add .bs to the version to help check we are on the right repo/branch

stas00 committed 4 years ago

bdef48f7

fix the deepspeed example

stas00 committed 4 years ago

1cef056c

Newer Older