bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits
[testing] fixes for pt-1.10 (#71)
stas00 committed 4 years ago
Verified a96e2ab3
Add openwebtext1000.jsonl to .gitignore
thomasw21 committed 4 years ago
9341269d
- Recompute bin/idx using microsoft/Megatron-DeepSpeed (Not changes)
thomasw21 committed 4 years ago
5eeae0b8
fix: exclusive scan computing pointers list (#68)
adammoody committed 4 years ago
Verified 191a96b0
shuffle index list with numpy, scatter list, use file for large lists (#63)
adammoody committed 4 years ago
Verified a59d93c0
fix: doc_idx offset when merging indexed dataset files (#66)
adammoody committed 4 years ago
Verified c680954e
fix header
stas00 committed 4 years ago
1c1423ed
add testing and contribute info
stas00 committed 4 years ago
0d404320
fix arg help (#65)
stas00 committed 4 years ago
Verified 56c2983b
Add test suite (#64)
stas00 committed 4 years ago
Verified 55b7036c
use HuggingFace Datasets as source to build Megatron data files (#48)
adammoody committed 4 years ago
Verified 5069622a
[microsoft/Megatron-DeepSpeed sync] Commits including 2021-08-09 (#58)
stas00 committed 4 years ago
Verified 3c9d748b
Add GLU variants (#47)
jaketae committed 4 years ago
Verified effb2fb8
better comment on TB writer (`is_last_rank`)
VictorSanh committed 4 years ago
5e3963db
Fix pretrain_gpt_single_node example script to have only one occurence of lr-decay-style
thomasw21 committed 4 years ago
7228b89e
chore: add deepspeed as comment
jaketae committed 4 years ago
Verified f4566b85
Update README.md (#51)
lintangsutawika committed 4 years ago
Verified a6cbb3bb
Add LRU cache, add faster tokenization (#37)
huu4ontocord committed 4 years ago
Verified 36284576
Update requirements.txt (#46)
jaketae committed 4 years ago
Verified f919d0cf
rm `(s)` that slipped through
VictorSanh committed 4 years ago
49b9d7de
Group tensorboard metrics (#39)
VictorSanh committed 4 years ago
Verified 9e75429d
chore: update gitignore (#45)
jaketae committed 4 years ago
Verified 752e958c
fix attn_mask (#50)
stas00 committed 4 years ago
Verified 42fe3b39
add .bs to the version to help check we are on the right repo/branch
stas00 committed 4 years ago
bdef48f7
fix the deepspeed example
stas00 committed 4 years ago
1cef056c
add a section on how we use deepspeed with Meg
stas00 committed 4 years ago
faf9809f
Faster preprocessing (#18)
thomasw21 committed 4 years ago
Verified 7b998814
Create README.md
stas00 committed 4 years ago
Verified 6394089f
Revert "Revert "use pp engine even for pp=1 (#6) (#34)""
stas00 committed 4 years ago
781676b5
Revert "use pp engine even for pp=1 (#6) (#34)"
stas00 committed 4 years ago
ad2e3d3a
Terms Of Use Privacy Policy Refund Policy Impressum