microsoft/onnxruntime

Pull Requests Commits

fix minor issue

pengwa committed 4 years ago

62ad731d

estimate memory usage by num params

pengwa committed 4 years ago

fc507965

refine the session init parallel size

pengwa committed 4 years ago

5fc7b21d

parallel session init as much as possible

pengwa committed 4 years ago

becba93f

fix dropout raiot = 0 , mlp matching

pengwa committed 4 years ago

53655af4

add max norm as user option

pengwa committed 4 years ago

7c4aa945

refine model export cache logic - phase 2

pengwa committed 4 years ago

01c924ac

refine large model export process - phase 1

pengwa committed 4 years ago

7735da6f

workaround onnx1.8 issue

pengwa committed 4 years ago

f7a76953

refine some envrionments, in case of user complexity

pengwa committed 4 years ago

3adaadc6

fix dropout in eval_step

pengwa committed 4 years ago

b655e241

wait for PT model GC before creating session, to aviod CPU memory exhausted

pengwa committed 4 years ago

8bb20939

load checkpoint

pengwa committed 4 years ago

1c2267fd

put first run graph in specified fodler; exp code commented for furthur investigation

pengwa committed 4 years ago

bb2a90f2

1. add index for all collective op ( to make sure they are running in the right order, instead of hanging there). 2. make megatron transformer only run once (to make sure the collective op is inserted with index increasing as topology order). 3. zero + mega 128 GPUs run fix, when no weight grad should be all-reduced, the grad_norm inputs are empty. so we pass zero.

pengwa committed 4 years ago

9c24416d

Introduce VariadicAlias, remove hardcoded alias limits (#6106)

SherlockNoMad committed 5 years ago

e37d99d4

hardcoded allreduce in fp32 - align with fairseq

pengwa committed 5 years ago

e3726e2b

[fix] clip_norm hard coded to 0.1

pengwa committed 5 years ago

a0c99e2b

Add moment/weight norm as outputs for debugging purpose; align loss scale with FairSeq

pengwa committed 5 years ago

9c06b9fd

scale grad with sample size before or after all reduce

pengwa committed 5 years ago

666496c0

Fix AllReduce kernel for contiguous buffer

Sherlock Huang committed 5 years ago

902d2fdd

Merge branch 'devdiv/bart' of https://github.com/microsoft/onnxruntime into devdiv/bart

pengwa committed 5 years ago

5b5a03fa

fix megatron bp computation, move sample divide before allreduce

pengwa committed 5 years ago

2588af86

Fix a bug in ort_trainer.py

Sherlock Huang committed 5 years ago

a9e5e27e

Enable Recompute for Bart

Sherlock Huang committed 5 years ago

6aecd6d1

Merge commit 'f54ac1b3c1881f7d31fc3fdbf4eb63167b2df12a' into devdiv/bart

Sherlock Huang committed 5 years ago

73e00cdf

fix megatron transformer

pengwa committed 5 years ago

3a77fa37

megatron l2norm, fix build

pengwa committed 5 years ago

aa6dc754

frontend changes

pengwa committed 5 years ago

b4b89d34

transformer changes

pengwa committed 5 years ago

fa6143d2

Older