bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits

update iteration

stas00 committed 4 years ago

8a7e9fa8

backport DebugUnderflowOverflow

stas00 committed 4 years ago

8327d44a

backport skip iterations range feature

stas00 committed 4 years ago

7a0158ef

Corby's numerically more stable self attn version (#118)

stas00 committed 4 years ago

Verified 48902f15

stas00 committed 4 years ago

1d9c2b51

Check cardon directory is not None (#74)

thomasw21 committed 4 years ago

39e9665f

[codecarbon] integration (#15)

stas00 committed 4 years ago

88c45191

Log timers fix (#59)

stas00 committed 4 years ago

Verified afb1a4c8

[microsoft/Megatron-DeepSpeed sync] Commits including 2021-08-09 (#58)

stas00 committed 4 years ago

277185f8

fix attn_mask (#50)

stas00 committed 4 years ago

9fc70acb

add a section on how we use deepspeed with Meg

stas00 committed 4 years ago

d0d4f4d4

Faster preprocessing (#18)

thomasw21 committed 4 years ago

Verified 7b998814

Create README.md

stas00 committed 4 years ago

Verified 6394089f

Revert "Revert "use pp engine even for pp=1 (#6) (#34)""

stas00 committed 4 years ago

781676b5

Revert "use pp engine even for pp=1 (#6) (#34)"

stas00 committed 4 years ago

ad2e3d3a

use pp engine even for pp=1 (#6) (#34)

stas00 committed 4 years ago

Verified 6c6c64ac

Revert "Update gpt2_tokenization.py"

thomasw21 committed 4 years ago

8938d7ac

Update gpt2_tokenization.py

huu4ontocord committed 4 years ago

Verified a40d8163

fix timing (#31)

stas00 committed 4 years ago

Verified feca57ac

Implement rotary embeddings (#7)

thomasw21 committed 4 years ago

Verified dc4e0cba

adding missing prefix on merge

stas00 committed 4 years ago

5563ec6f

[microsoft/Megatron-DeepSpeed sync] Commits including Jul 26, 2021 (#29)

stas00 committed 4 years ago

Verified 084c3d0c

Fix inconsistent merge (#30)

thomasw21 committed 4 years ago

Verified 0125aaa7

speed up preprocessing with workers

stas00 committed 4 years ago

fcd38360

improve setup doc

stas00 committed 4 years ago

55f17201

full setup instructions

stas00 committed 4 years ago

ee708b6b

Seperate merge script from process_data script (#19)

thomasw21 committed 4 years ago

Verified e8fcbae2

extend preprocess_data.py to support merging datasets (#13)

stas00 committed 4 years ago

Verified 18201ce4

fix bug when restarting with no eval in round 1 (#12)

stas00 committed 4 years ago

Verified 9189c4eb

Revert "fix bug when restarting with no eval in round 1 (#11)"

stas00 committed 4 years ago

e0fb5ab0

Older