DeepSpeed
Hybrid Engine Refactor and Llama Inference Support
#3425
Merged
Overview
Commits
44
Changes
View On GitHub

Hybrid Engine Refactor and Llama Inference Support #3425

jeffra merged 44 commits into master from cholmes/llama-inference-v2
cmikeh2
cmikeh2 Rebase changes onto correct git history
21b0540e
cmikeh2 Further generalize rotate half rotary position embeddings
bd74c32a
cmikeh2 Upgrade local clang-format to match CI
56e1de9b
cmikeh2 Restore GeGLU behavior and template for SiLU. Add unit test.
264d49b3
cmikeh2 Restore experimental qkv reset
14c6a9b7
cmikeh2 Switch to named constant to improve readability
b01e7ea7
cmikeh2 Name refactor to align with functionality rather than implementation
0215867e
cmikeh2 Finish hybrid engine integration
0f371d84
cmikeh2 Fix for MLP dimensions
48db17b4
cmikeh2 Refactor changes out of base container (#506)
ce19b9c8
cmikeh2 Update explanations
88a821df
cmikeh2 Complete merge, fix BF16 integration
e42859bd
cmikeh2 Merge master
6bad46d1
cmikeh2 BF16_AVAILABLE should derive solely from the op_builder
eebfcdf3
cmikeh2 Merge remote-tracking branch 'public/master' into cholmes/llama-infer…
6ad5f0f7
cmikeh2 Refactor on top of additional model support
63fe26ff
cmikeh2 cmikeh2 requested a review from jeffra jeffra 3 years ago
cmikeh2 cmikeh2 requested a review from tjruwase tjruwase 3 years ago
cmikeh2 cmikeh2 requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 3 years ago
cmikeh2 cmikeh2 requested a review from mrwyattii mrwyattii 3 years ago
cmikeh2 cmikeh2 requested a review from awan-10 awan-10 3 years ago
cmikeh2 cmikeh2 requested a review from arashb arashb 3 years ago
cmikeh2 Guard is_bf16_supported check
e9137d18
cmikeh2 Even stronger guards
838b6f4b
cmikeh2 Remove deprecated policy members
a496dd19
cmikeh2 Merge branch 'master' into cholmes/llama-inference-v2
123e195a
cmikeh2 Another guard
c6703440
cmikeh2 Merge branch 'cholmes/llama-inference-v2' of github.com:microsoft/Dee…
6830f5a5
cmikeh2 Bad check for BF16 support
708fb456
cmikeh2 Merge fix
ace19679
cmikeh2 Call correct parent func
439f26f3
cmikeh2 Revert API change for TP
81c8e629
cmikeh2 Reorder inheritance for Hybrid containers, provide justification
80a2c67b
cmikeh2 Add missing policy kwargs
4bd0c6e8
cmikeh2 Merge remote-tracking branch 'public/master' into cholmes/llama-infer…
a974832a
cmikeh2 Remove debug code
775282a8
awan-10
awan-10 commented on 2023-05-02
awan-10
awan-10 commented on 2023-05-02
awan-10
awan-10 commented on 2023-05-02
awan-10
awan-10 commented on 2023-05-02
awan-10
awan-10 commented on 2023-05-02
awan-10
awan-10 approved these changes on 2023-05-02
cmikeh2 BF16 model inference support
9725f09e
cmikeh2 Remove debug code
e0e70fe9
cmikeh2 Review feedback
9d645153
jeffra
jeffra approved these changes on 2023-05-02
jeffra Merge branch 'master' into cholmes/llama-inference-v2
c94c67bc
lekurile
lekurile approved these changes on 2023-05-02
cmikeh2 Fix inheritance
79ad7d0a
jeffra jeffra enabled auto-merge (squash) 3 years ago
cmikeh2 Merge branch 'cholmes/llama-inference-v2' of github.com:microsoft/Dee…
a044359b
disabled auto-merge 3 years ago
Manually disabled by user
jeffra don't use cache dir for torch installs
5245e0a4
jeffra jeffra requested a review from loadams loadams 3 years ago
jeffra Merge branch 'master' into cholmes/llama-inference-v2
8c10a296
loadams
loadams approved these changes on 2023-05-03
cmikeh2 Align APIs
cd1b6176
cmikeh2 Merge branch 'cholmes/llama-inference-v2' of github.com:microsoft/Dee…
149e1937
jeffra add HE unit test for OPT
8adc056d
jeffra fix typo, missing policy ref to client module
995c3ae2
jeffra Merge branch 'master' into cholmes/llama-inference-v2
1596640a
jeffra Merge branch 'master' into cholmes/llama-inference-v2
d124817a
jeffra jeffra merged 0a61d5d6 into master 3 years ago
jeffra jeffra deleted the cholmes/llama-inference-v2 branch 3 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum