DeepSpeed
Inference Refactor (replace_with_policy, model_implementations)
#2554
Merged
Overview
Commits
69
Changes
View On GitHub

Inference Refactor (replace_with_policy, model_implementations) #2554

jeffra merged 69 commits into master from inference-refactor-v1
awan-10
awan-10 Public inference v2 (#387)
102a1bdb
awan-10 remove notes.
3f511079
awan-10 awan-10 assigned lekurile lekurile 3 years ago
lekurile Container based OPT is working
8ca77499
lekurile lekurile requested a review from lekurile lekurile 3 years ago
lekurile Megatron container working w/ moe=1
75003bed
lekurile Add ds_megatron_gpt model implementation + make explicit megatron is …
9a36258d
lekurile Clean up
09f37abb
lekurile Merge branch 'master' into inference-refactor-v1
74233437
lekurile TODO comments
9b24af91
lekurile Update megatron container to work with MoE==1 text-generation example
d190f68f
lekurile move _transpose() to base container
5651f9e3
lekurile Add megatron_v2 class variable to megatron policy
8186da32
lekurile Change _transpose to transpose_qkv_alignment
699336bb
lekurile Properly source megatron_v2 flag from policy in containers
eecae6fe
lekurile Add comprehension of megatron_v2 in gptneox, and rotation config
992ace22
lekurile remove TODO
28e6b617
lekurile Make containers inherit scale_attention from policy
105ba4b3
lekurile Merge branch 'master' into inference-refactor-v1
4479a07b
lekurile Update all policies for generic load checkpoint
4e668887
lekurile Add generic load checkpoint to replace_layer, still need to add to co…
79643351
lekurile Make megatron_v2 part of base container
52bba028
lekurile Update from model_config to config in replace_module.py
3feff1fd
lekurile Merge branch 'master' into inference-refactor-v1
55e929df
lekurile Get generic checkpoint loading working w/o containers
a8f440e3
lekurile Use self. for tensors in bloom container
7b65351a
lekurile Comprehend quantizer in containers + generic checkpoint works for GPT…
794092e3
lekurile Remove .scale comments from ds_mlp and ds_attention
fbd0b65a
lekurile Whitespace in replace_layer to match replace_module
bc9002ae
lekurile Apply weight quantization in BERT container
791935ca
lekurile Small clean-up
da475804
lekurile Move weight quantization into separate function in base container
381e61b3
lekurile First base container re-org
a0979fea
lekurile Fix a few bugs after the base re-org
5d602d62
lekurile Remove unnecessary code from BERT container
e71035d2
mrwyattii moved megatron and meta tensor support to separate containers, added …
1d5149d3
lekurile Add MegatronContainer to GPTNEOX container
22d7c8c1
lekurile Move feature containers into features folder
8136d71a
lekurile Update BLOOM container
7e03935c
lekurile Remove gpt2 model container transpose, since base comprehends guard f…
bb297022
lekurile Add __init__.py in features folder
ede2749c
lekurile Merge branch 'master' into inference-refactor-v1
2294cb8d
lekurile Merge branch 'master' into inference-refactor-v1
b8527530
lekurile Add MoE container (still needs testing) + various refactor
7884c9c6
lekurile Merge branch 'master' into inference-refactor-v1
b368af45
lekurile Merge branch 'master' into inference-refactor-v1
ae313969
lekurile Fix syntax/import for base moe container
6cc71070
lekurile Source config params instead of using magic numbers
a6a807be
lekurile Add DistilBERT to utils.py
aff6cc3a
lekurile Create MoE config function
fa56b96b
lekurile Use **kwargs for containers
5a2c192b
lekurile Remove init_inference_v2 function, use getattr in place of hasattr
32180da6
lekurile Merge branch 'master' into inference-refactor-v1
711231d5
lekurile Move replace_module components into replace_layer, part of consolidation
fc921397
lekurile Move policies + containers into same file
b1edf4fc
lekurile Update policy imports
3a6ff98c
lekurile Remove separate policy files since policies consolidated in container…
dac16023
lekurile Update replace_layer w/ code from replace_module
fa437de3
lekurile Make replace_layer -> replace_module, update to match replace_module …
a1122d61
lekurile Update minor policy changes from master
58b1e374
lekurile container fixes for SD models
c98c757d
lekurile Clean up and polish
6a814bd8
lekurile print_configuration to use logger
cedd3d59
lekurile Remove debug prints from policy_to_ds_container
b8844799
lekurile lekurile changed the title Inference v2 Inference Refactor (replace_with_policy, model_implementations) 2 years ago
lekurile Merge branch 'master' into inference-refactor-v1
24984334
lekurile lekurile marked this pull request as ready for review 2 years ago
lekurile lekurile requested a review from jeffra jeffra 2 years ago
lekurile lekurile requested a review from tjruwase tjruwase 2 years ago
lekurile lekurile requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 2 years ago
lekurile lekurile requested a review from mrwyattii mrwyattii 2 years ago
lekurile lekurile requested a review from cmikeh2 cmikeh2 2 years ago
lekurile lekurile requested a review from arashb arashb 2 years ago
lekurile Merge branch 'master' into inference-refactor-v1
bfeba271
lekurile Update CLIP policy
ed09e065
mrwyattii
mrwyattii requested changes on 2023-01-18
mrwyattii Merge branch 'master' into inference-refactor-v1
3ca419d6
mrwyattii refactor policy to container mapping function
3a2fa502
lekurile Use @abstractmethod on policy functions
c6a4c50b
lekurile Merge branch 'master' into inference-refactor-v1
cdca7be5
mrwyattii
mrwyattii approved these changes on 2023-01-19
lekurile
lekurile approved these changes on 2023-01-19
jeffra
jeffra approved these changes on 2023-01-19
jeffra jeffra merged 867da307 into master 2 years ago
jeffra jeffra deleted the inference-refactor-v1 branch 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum