bigscience-workshop/Megatron-DeepSpeed

Pull Requests Commits

Port MTF changes

Muennighoff committed 3 years ago

a1d9317d

Merge pull request #1 from bigscience-workshop/lumi_eval

spyysalo committed 3 years ago

Verified fdd57c4b

Muennighoff committed 3 years ago

32f039c2

Add LUMI eval compat

Muennighoff committed 3 years ago

2963caea

Revert cherry-picked changes to .py

spyysalo committed 3 years ago

277e1d38

Fix the bug of FusedLayerNorm on ROCm (#96)

hubertlu-tw committed 3 years ago

9b7cd052

Bugfix (thanks to Thomas Wang for catching this)

spyysalo committed 3 years ago

18e2c65b

Add --no-optimizer-fusion argument

spyysalo committed 3 years ago

e0487132

Add --no-layer-norm-fusion argument

spyysalo committed 3 years ago

21c90de1

Squash 3 commits to 1

luukkonenr committed 3 years ago

ebb79c86

relocating to https://github.com/huggingface/transformers-bloom-inference

stas00 committed 3 years ago

09a35f53

[bloom inference scripts] improvements (#345)

stas00 committed 3 years ago

Verified 4a7bb886

Followup PR for adding generation-server (#339)

mayank31398 committed 3 years ago

Verified cd597c8f

[ds-inference bloom] tweaks (#340)

stas00 committed 3 years ago

Verified 479aac39

Add generation server scripts using HF accelerate and DS-inference (#328)

mayank31398 committed 3 years ago

Verified f9402d02

disable CI (#332)

stas00 committed 3 years ago

Verified c1139c70

BLOOM Inference via DeepSpeed-Inference, Accelerate and DeepSpeed-ZeRO (#308)

stas00 committed 3 years ago

Verified 3932c749

Reshape deepspeed checkpoint (#239)

tjruwase committed 3 years ago

Verified 0f23a729

not yet working script

stas00 committed 3 years ago

7b5f175b

Create README.md

stas00 committed 3 years ago

Verified 2ce8bb4f

Fix causal attention mask (#306)

thomasw21 committed 3 years ago

Verified 38607ae9

Add bias a weight we need to sync as well (#307)

thomasw21 committed 3 years ago

Verified 0d0d84c8

Combine Specs (#304)

Muennighoff committed 3 years ago

Verified c3be5d3f

Add support for weighted train (#299)

thomasw21 committed 3 years ago

Verified 43ab0e08

MTF train script (#295)

thomasw21 committed 3 years ago

Verified 3d5d1514

sync layer norms (#272)

stas00 committed 3 years ago

Verified e1c479e5

CI fixes (#302)

stas00 committed 3 years ago

Verified 0cb043cf

MTF dataset and packing (#293)

thomasw21 committed 3 years ago

Verified c5b88fb9

Merge MLM too fast 2 (#294)

thomasw21 committed 3 years ago

Verified 131bd43e

Eval harness (#212)

DanielHesslow committed 3 years ago

Verified 3ab0ad18

Older