bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits

corby's numerically mmore stable self attn version

stas00 committed 4 years ago

e06841cf

ALiBi Implementation (#101)

ofirpress committed 4 years ago

Verified c839a8aa

stas00 committed 4 years ago

2f662e84

make scripts executable

stas00 committed 4 years ago

26f18b5a

update merge_preprocessed_data to use distributed merge (#82)

adammoody committed 4 years ago

Verified 0c820648

training with dummy data to verify sampling (#36)

lintangsutawika committed 4 years ago

Verified 28fd70ad

add direct meg-ds to hf format script (part2) (#111)

stas00 committed 4 years ago

Verified d6c2a803

add direct meg-ds to hf format script (#110)

stas00 committed 4 years ago

Verified bea5ded0

Checkpoint conversion tools (#14) (#109)

stas00 committed 4 years ago

Verified bc9c951a

[tensor comparisons] support pt-1.8, add torch_assert_close (#106)

stas00 committed 4 years ago

Verified fc8820ca

Fix model tests (#103)

thomasw21 committed 4 years ago

Verified 48dcee4a

simplify the CI trigger (#102)

stas00 committed 4 years ago

Verified 0f82f40b

Prefix lm (#52)

thomasw21 committed 4 years ago

Verified 68b46f20

check whether python3-config is available (#98)

stas00 committed 4 years ago

Verified 648ee17f

TevenLeScao committed 4 years ago

a7609226

Floating-point ops counting and reloading (#40)

TevenLeScao committed 4 years ago

Verified af8229e2

[WIP] [codecarbon] sorting out CC warnings + logger preamble (#80)

stas00 committed 4 years ago

Verified 7cd2b85a

[requirements] fix format (#94)

stas00 committed 4 years ago

Verified a42a398a

[CI] setting up a CI with EC2 backend (#78)

stas00 committed 4 years ago

Verified 9e14c02a

add missing dependencies (#88)

stas00 committed 4 years ago

Verified b4cc77fe

dynamically use as many 3d dimensions as possible (#83)

stas00 committed 4 years ago

Verified a025c8a1

Add step to download dataset before running the preprocess_data_dist test (#81)

adammoody committed 4 years ago

Verified fa9f54c9

fix test; skip broken test (#79)

stas00 committed 4 years ago

Verified 8427ca8f

distributed merge of per-rank Megatron data files (#55)

adammoody committed 4 years ago

Verified 97221116

stas00 committed 4 years ago

Verified 6d88ae20

[CI] start workflow (#75)

stas00 committed 4 years ago

Verified 3fd48db5

Check cardon directory is not None (#74)

thomasw21 committed 4 years ago

Verified e96df7dc

[codecarbon] integration (#15)

stas00 committed 4 years ago

Verified 350fb903

fix circular import (#72)

stas00 committed 4 years ago

Verified 4255845d

Expose GLU activations as arguments (#69)

jaketae committed 4 years ago

Verified b5a029d9

Older