DeepSpeed
Add `scale_attn_by_inverse_layer_idx` feature
#2486
Merged
Overview
Commits
5
Changes
View On GitHub

Add `scale_attn_by_inverse_layer_idx` feature #2486

RezaYazdaniAminabadi merged 5 commits into deepspeedai:master from master
hyunwoongko
hyunwoongko Add scale_attn_by_inverse_layer_idx feature
4379dea5
hyunwoongko Fix layer_id bug
a77a1739
hyunwoongko Fix scaling value
82ed33e8
hyunwoongko hyunwoongko requested a review from jeffra jeffra 4 years ago
hyunwoongko hyunwoongko requested a review from samyam samyam 4 years ago
hyunwoongko hyunwoongko requested a review from tjruwase tjruwase 4 years ago
hyunwoongko hyunwoongko requested a review from ShadenSmith ShadenSmith 4 years ago
hyunwoongko hyunwoongko requested a review from conglongli conglongli 4 years ago
hyunwoongko hyunwoongko requested a review from awan-10 awan-10 4 years ago
hyunwoongko hyunwoongko requested a review from cli99 cli99 4 years ago
hyunwoongko hyunwoongko requested a review from eltonzheng eltonzheng 4 years ago
hyunwoongko hyunwoongko requested a review from minjiaz minjiaz 4 years ago
hyunwoongko hyunwoongko requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 4 years ago
hyunwoongko hyunwoongko requested a review from duli2012 duli2012 4 years ago
hyunwoongko hyunwoongko requested a review from mrwyattii mrwyattii 4 years ago
hyunwoongko hyunwoongko requested a review from yaozhewei yaozhewei 4 years ago
hyunwoongko hyunwoongko requested a review from arashb arashb 4 years ago
hyunwoongko hyunwoongko requested a review from xiaoxiawu-microsoft xiaoxiawu-microsoft 4 years ago
hyunwoongko hyunwoongko requested a review from samadejacobs samadejacobs 4 years ago
hyunwoongko hyunwoongko requested a review from cmikeh2 cmikeh2 4 years ago
hyunwoongko hyunwoongko requested a review from GuanhuaWang GuanhuaWang 4 years ago
hyunwoongko
RezaYazdaniAminabadi
RezaYazdaniAminabadi
RezaYazdaniAminabadi approved these changes on 2022-11-08
cmikeh2 Merge branch 'master' into master
ceec339a
cmikeh2
cmikeh2 approved these changes on 2022-11-08
hyunwoongko
RezaYazdaniAminabadi Merge branch 'master' into master
0143dbe1
RezaYazdaniAminabadi RezaYazdaniAminabadi merged 6f77da1b into master 4 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum