DeepSpeed
DeepSpeed inference config. (#2459)
#2472
Merged
Overview
Commits
19
Changes
View On GitHub

DeepSpeed inference config. (#2459) #2472

mrwyattii merged 19 commits into master from staging-inf-config-v1
awan-10
awan-10 Prototype DS inference config. Tested with gpt2/bert. (#2459)
1b0c2fb5
awan-10 Merge branch 'master' into staging-inf-config-v1
0cc4fb7b
awan-10 add the missing max tokens fix.
7f192540
awan-10 awan-10 marked this pull request as ready for review 3 years ago
awan-10 awan-10 requested a review from jeffra jeffra 3 years ago
awan-10 awan-10 requested a review from samyam samyam 3 years ago
awan-10 awan-10 requested a review from tjruwase tjruwase 3 years ago
awan-10 awan-10 requested a review from ShadenSmith ShadenSmith 3 years ago
awan-10 awan-10 requested a review from conglongli conglongli 3 years ago
awan-10 awan-10 requested a review from cli99 cli99 3 years ago
awan-10 awan-10 requested a review from eltonzheng eltonzheng 3 years ago
awan-10 awan-10 requested a review from minjiaz minjiaz 3 years ago
awan-10 awan-10 requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 3 years ago
awan-10 awan-10 requested a review from duli2012 duli2012 3 years ago
awan-10 awan-10 requested a review from mrwyattii mrwyattii 3 years ago
awan-10 awan-10 requested a review from yaozhewei yaozhewei 3 years ago
awan-10 awan-10 requested a review from arashb arashb 3 years ago
awan-10 awan-10 requested a review from xiaoxiawu-microsoft xiaoxiawu-microsoft 3 years ago
awan-10 awan-10 requested a review from samadejacobs samadejacobs 3 years ago
awan-10 awan-10 requested a review from cmikeh2 cmikeh2 3 years ago
awan-10 awan-10 requested a review from GuanhuaWang GuanhuaWang 3 years ago
awan-10 fix format.
69db92d8
awan-10 Merge branch 'master' into staging-inf-config-v1
8f396386
awan-10 moe fixes.
72c0b99a
awan-10 Merge branch 'master' into staging-inf-config-v1
8f758b6b
mrwyattii Merge branch 'master' into staging-inf-config-v1
bc3af199
mrwyattii fix mp_size validation not properly setting tp_size
61380387
mrwyattii fix broken custom injection policy
36a53c9c
awan-10 Merge branch 'master' into staging-inf-config-v1
c7dbb33f
awan-10 Merge branch 'master' into staging-inf-config-v1
399f45b5
awan-10 first round of config to be passed to apply_injection_policy. manual
ac95ab7f
awan-10 fix several things.
38bb2088
awan-10 format fix
b3b451c2
awan-10 Merge branch 'master' into staging-inf-config-v1
cc7ccb5f
awan-10 awan-10 changed the title Prototype DS inference config. Tested with gpt2/bert. (#2459) DeepSpeed inference config. (#2459) 3 years ago
awan-10
awan-10 Merge branch 'master' into staging-inf-config-v1
bbe351f5
cmikeh2
cmikeh2 approved these changes on 2022-11-14
mrwyattii moved docs to appropriate place in config model
266e60bd
mrwyattii added bf16 support for dtype
a3d70c22
mrwyattii
mrwyattii approved these changes on 2022-11-14
mrwyattii mrwyattii enabled auto-merge (squash) 3 years ago
mrwyattii mrwyattii merged b5d18a6a into master 3 years ago
awan-10 awan-10 assigned awan-10 awan-10 3 years ago
awan-10 awan-10 assigned mrwyattii mrwyattii 3 years ago
mrwyattii mrwyattii deleted the staging-inf-config-v1 branch 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum