DeepSpeed
Add local attention for GPT-Neo model architecture
#1114
Merged
Overview
Commits
8
Changes
View On GitHub

Add local attention for GPT-Neo model architecture #1114

jeffra merged 8 commits into master from reyazda/add-local-attention
RezaYazdaniAminabadi
fix links for inference tutorial
a7c29bab
Fix automatic injection. Add the local-attention for GPT-Neo
1db4625a
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from arashashari arashashari 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from awan-10 awan-10 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from cli99 cli99 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from conglongli conglongli 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from eltonzheng eltonzheng 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from jeffra jeffra 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from minjiaz minjiaz 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from niumanar niumanar 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from samyam samyam 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from ShadenSmith ShadenSmith 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from tjruwase tjruwase 4 years ago
RezaYazdaniAminabadi Merge branch 'master' into reyazda/add-local-attention
59555f2a
RezaYazdaniAminabadi RezaYazdaniAminabadi changed the title Reyazda/add local attention Add local attention for GPT-Neo model architecture 4 years ago
jeffra Merge branch 'master' into reyazda/add-local-attention
c6e75f00
fix the inference for generation of large sequences (>1K & <32K)
5e9b420e
Merge branch 'reyazda/add-local-attention' of github.com:microsoft/De…
56561d49
fix format
800faabb
jeffra Merge branch 'master' into reyazda/add-local-attention
de614e6d
jeffra
jeffra approved these changes on 2021-06-08
jeffra jeffra merged aca7fc54 into master 4 years ago
jeffra jeffra deleted the reyazda/add-local-attention branch 4 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum