Floating-point ops counting and reloading #40
TevenLeScao
merged
24 commits into
main
from
training_flos
initial flo count/logging setup (need to fix model parameter count)
5196f8a5
initial flo count/logging setup (need to fix model parameter count)
a3a4ba46
1B3 parameter setup + flos counting
bdd75f18
1B3 parameter setup + flos counting
c0fc29a6
1B3 parameter setup + flos counting
aefbe3bd
1B3 parameter setup
17e01842
1B3 parameter setup
97dd06db
synched with latest 13B script
64892e27
synched with latest 13B script
3c79aaca
pipe transformer docstring
b7b3167f
improve DS integration evaluation + logging
8382141c
use pp engine even for pp=1 (#6)
06cb18fa
removed slurm_examples
d5818947
flos re-loading
60794bf7
TevenLeScao
commented
on 2021-08-04
TevenLeScao
commented
on 2021-08-04
TevenLeScao
commented
on 2021-08-04
TevenLeScao
commented
on 2021-08-04
thomasw21
commented
on 2021-08-04
Merge branch 'main' into training_flos
c79db1cd
Update megatron/training.py
fb33f138
Update megatron/data/gpt_dataset.py
dff1479b
stas00
commented
on 2021-08-24
Update megatron/utils.py
2fa3b5b8
Update megatron/utils.py
ff7af108
formatting fix, reserving bug for somewhere else, adding flo-logging …
b9ac381f
indentation bug
f25e25f5
fixing possible double counts
e63503dd
stas00
commented
on 2021-08-25
stas00
commented
on 2021-08-25
tweaks
5bdcf819
warning for double counts
72ad7113
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub