bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits
1B3 parameter setup + flos counting
TevenLeScao committed 4 years ago
c0fc29a6
1B3 parameter setup + flos counting
TevenLeScao committed 4 years ago
bdd75f18
speed up preprocessing with workers
stas00 committed 4 years ago
fcd38360
improve setup doc
stas00 committed 4 years ago
55f17201
full setup instructions
stas00 committed 4 years ago
ee708b6b
Seperate merge script from process_data script (#19)
thomasw21 committed 4 years ago
Verified e8fcbae2
extend preprocess_data.py to support merging datasets (#13)
stas00 committed 4 years ago
Verified 18201ce4
fix bug when restarting with no eval in round 1 (#12)
stas00 committed 4 years ago
Verified 9189c4eb
Revert "fix bug when restarting with no eval in round 1 (#11)"
stas00 committed 4 years ago
e0fb5ab0
remove the forked limitation note
stas00 committed 4 years ago
Verified 8d103789
fix bug when restarting with no eval in round 1 (#11)
stas00 committed 4 years ago
Verified 63671cb4
Revert "train_samples check fix"
stas00 committed 4 years ago
0e78387d
train_samples check fix
stas00 committed 4 years ago
d13f5c6b
Fixed single node test scripts to use DS
TevenLeScao committed 4 years ago
Verified a7a534ae
Fixed single node test scripts to use DS
TevenLeScao committed 4 years ago
Verified 1556d7e6
initial flo count/logging setup (need to fix model parameter count)
TevenLeScao committed 4 years ago
a3a4ba46
adjust the script
stas00 committed 4 years ago
5ab4fad9
add example script for one node
stas00 committed 4 years ago
2048ca57
update docs
stas00 committed 4 years ago
12503bd2
consistency
stas00 committed 4 years ago
8cf27f22
install instructions; consistent indent
stas00 committed 4 years ago
7d880b7c
update to deepspeed usage; rm irrelevant info
stas00 committed 4 years ago
c5e0b248
initial flo count/logging setup (need to fix model parameter count)
TevenLeScao committed 4 years ago
5196f8a5
Tiny gpt training script for local testing
TevenLeScao committed 4 years ago
Verified d61ca5c2
HF tokenizers (on the training side) (#5)
TevenLeScao committed 4 years ago
Verified 0b2f0df4
Update README.md
TevenLeScao committed 4 years ago
Verified ca4ea155
Update README.md
TevenLeScao committed 4 years ago
Verified a0ab8f48
Create README for preprocessing
TevenLeScao committed 4 years ago
Verified ebb6a078
Hf tok pipeline (#2)
sbmaruf committed 4 years ago
Verified b1a30de8
add the explanation about the forking problems.
stas00 committed 4 years ago
Verified 5502865b
Terms Of Use Privacy Policy Refund Policy Impressum