microsoft/DeepSpeed

Pull Requests Commits

moved megatron and meta tensor support to separate containers, added multi-class inheritance

mrwyattii committed 2 years ago

0c4b726a

Fix a few bugs after the base re-org

lekurile committed 2 years ago

5d602d62

First base container re-org

lekurile committed 2 years ago

a0979fea

Move weight quantization into separate function in base container

lekurile committed 2 years ago

381e61b3

lekurile committed 2 years ago

da475804

Apply weight quantization in BERT container

lekurile committed 2 years ago

791935ca

Whitespace in replace_layer to match replace_module

lekurile committed 2 years ago

bc9002ae

Remove .scale comments from ds_mlp and ds_attention

lekurile committed 2 years ago

fbd0b65a

Comprehend quantizer in containers + generic checkpoint works for GPTJ w/ container

lekurile committed 2 years ago

794092e3

Use self. for tensors in bloom container

lekurile committed 2 years ago

7b65351a

Get generic checkpoint loading working w/o containers

lekurile committed 2 years ago

a8f440e3

Merge branch 'master' into inference-refactor-v1

lekurile committed 2 years ago

55e929df

jeffra committed 2 years ago

2600db54

get mask token from tokenizer (#2592)

jeffra committed 2 years ago

Verified 2076bf23

Update from model_config to config in replace_module.py

lekurile committed 2 years ago

3feff1fd

Fix issues w. python 3.6 + add py-version checks to CI (#2589)

jeffra committed 2 years ago

Verified 35eabb0a

Make megatron_v2 part of base container

lekurile committed 2 years ago

52bba028

Updating API docs (#2586)

jomayeri committed 2 years ago

Verified 18713c68

Updating docs README (#2587)

jomayeri committed 2 years ago

Verified 377c770a

Add generic load checkpoint to replace_layer, still need to add to containers

lekurile committed 2 years ago

79643351

Update all policies for generic load checkpoint

lekurile committed 2 years ago

4e668887

Merge branch 'master' into inference-refactor-v1

lekurile committed 2 years ago

4479a07b

Add checkpoint sharding unit tests (#2561)

mrwyattii committed 2 years ago

Verified ccb8eb81

Support N-dimension input in quantization kernel (#2575)

lokoppakmsft committed 2 years ago

Verified 591744eb

Make containers inherit scale_attention from policy

lekurile committed 2 years ago

105ba4b3

lekurile committed 2 years ago

28e6b617

Add comprehension of megatron_v2 in gptneox, and rotation config

lekurile committed 2 years ago

992ace22

Properly source megatron_v2 flag from policy in containers

lekurile committed 2 years ago

eecae6fe

Change _transpose to transpose_qkv_alignment

lekurile committed 2 years ago

699336bb

Update barrier and reduce_scatter_base to conform to PyTorch signatures (#2570)

Quentin-Anthony committed 2 years ago

Verified 18d55e54

Older