PR #2623 [inference] ds-attention refactor w.r.t. ops

[inference] ds-attention refactor w.r.t. ops #2623

jeffra merged 17 commits into master from jeffra/op-shim

refactor of ds-attn to use new op bindings

4c6a2574

Merge branch 'master' into jeffra/op-shim

4436042c

fix imports

dd6088f6

fix for softmax

8c1f1aab

fix typo

e7610c43

jeffra marked this pull request as ready for review 3 years ago

jeffra requested a review from

RezaYazdaniAminabadi 3 years ago

jeffra requested a review from

mrwyattii 3 years ago

jeffra requested a review from

awan-10 3 years ago

jeffra requested a review from

cmikeh2 3 years ago

jeffra requested a review from

arashb 3 years ago

jeffra changed the title ~~Inference backend refactor~~ [inference] ds-attention refactor w.r.t. ops 3 years ago

cmikeh2 commented on 2022-12-19

Merge branch 'master' into jeffra/op-shim

3be21698

cmikeh2 commented on 2022-12-19

address comments and consolidate imports

cabf95a7

Merge branch 'jeffra/op-shim' of github.com:microsoft/DeepSpeed into …

f413336c

fix import issue

804e1b94

move bloom specific attn into its own class

7a845df9

cmikeh2 approved these changes on 2022-12-21

remove dead code

ba1ffafc

Merge branch 'master' into jeffra/op-shim

7eec8848

Merge branch 'master' into jeffra/op-shim

7c0c1dcf

Merge branch 'master' into jeffra/op-shim

f722064e

Merge branch 'master' into jeffra/op-shim

ba0690c2

move softmax op to BloomSelfAttention

3b239335

only load op in base to avoid excessive logging

3b99b579

jeffra merged bb68c526 into master 3 years ago

jeffra deleted the jeffra/op-shim branch 3 years ago

Reviewers

cmikeh2

RezaYazdaniAminabadi

mrwyattii

awan-10

arashb

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed [inference] ds-attention refactor w.r.t. ops #2623 Merged

[inference] ds-attention refactor w.r.t. ops #2623

DeepSpeed
[inference] ds-attention refactor w.r.t. ops
#2623

Merged