DeepSpeed
Add `scale_attn_by_inverse_layer_idx` feature
#2486
Merged
Overview
Commits
5
Changes
View On GitHub

Add `scale_attn_by_inverse_layer_idx` feature #2486

RezaYazdaniAminabadi merged 5 commits into deepspeedai:master from master
hyunwoongko
hyunwoongko Add scale_attn_by_inverse_layer_idx feature
4379dea5
hyunwoongko Fix layer_id bug
a77a1739
hyunwoongko Fix scaling value
82ed33e8
hyunwoongko hyunwoongko requested a review from jeffra jeffra 3 years ago
hyunwoongko hyunwoongko requested a review from samyam samyam 3 years ago
hyunwoongko hyunwoongko requested a review from tjruwase tjruwase 3 years ago
hyunwoongko hyunwoongko requested a review from ShadenSmith ShadenSmith 3 years ago
hyunwoongko hyunwoongko requested a review from conglongli conglongli 3 years ago
hyunwoongko hyunwoongko requested a review from awan-10 awan-10 3 years ago
hyunwoongko hyunwoongko requested a review from cli99 cli99 3 years ago
hyunwoongko hyunwoongko requested a review from eltonzheng eltonzheng 3 years ago
hyunwoongko hyunwoongko requested a review from minjiaz minjiaz 3 years ago
hyunwoongko hyunwoongko requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 3 years ago
hyunwoongko hyunwoongko requested a review from duli2012 duli2012 3 years ago
hyunwoongko hyunwoongko requested a review from mrwyattii mrwyattii 3 years ago
hyunwoongko hyunwoongko requested a review from yaozhewei yaozhewei 3 years ago
hyunwoongko hyunwoongko requested a review from arashb arashb 3 years ago
hyunwoongko hyunwoongko requested a review from xiaoxiawu-microsoft xiaoxiawu-microsoft 3 years ago
hyunwoongko hyunwoongko requested a review from samadejacobs samadejacobs 3 years ago
hyunwoongko hyunwoongko requested a review from cmikeh2 cmikeh2 3 years ago
hyunwoongko hyunwoongko requested a review from GuanhuaWang GuanhuaWang 3 years ago
hyunwoongko
RezaYazdaniAminabadi
RezaYazdaniAminabadi
RezaYazdaniAminabadi approved these changes on 2022-11-08
cmikeh2 Merge branch 'master' into master
ceec339a
cmikeh2
cmikeh2 approved these changes on 2022-11-08
hyunwoongko
RezaYazdaniAminabadi Merge branch 'master' into master
0143dbe1
RezaYazdaniAminabadi RezaYazdaniAminabadi merged 6f77da1b into master 3 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum