DeepSpeed
Add Unidirectional Sparse Attention Type to BigBird and BSLongformer
#1959
Merged
Overview
Commits
3
Changes
View On GitHub

Add Unidirectional Sparse Attention Type to BigBird and BSLongformer #1959

jeffra merged 3 commits into master from qanthony/bigbird
Quentin-Anthony
Quentin-Anthony Add unidirectional attention options to BigBird and BSLongformer
be507383
Quentin-Anthony Added checks for bigbird config
e3705b68
Quentin-Anthony Quentin-Anthony requested a review from jeffra jeffra 3 years ago
Quentin-Anthony Quentin-Anthony requested a review from samyam samyam 3 years ago
Quentin-Anthony Quentin-Anthony requested a review from tjruwase tjruwase 3 years ago
Quentin-Anthony Quentin-Anthony requested a review from ShadenSmith ShadenSmith 3 years ago
Quentin-Anthony Quentin-Anthony requested a review from conglongli conglongli 3 years ago
Quentin-Anthony Quentin-Anthony requested a review from awan-10 awan-10 3 years ago
Quentin-Anthony Quentin-Anthony requested a review from cli99 cli99 3 years ago
Quentin-Anthony Quentin-Anthony requested a review from eltonzheng eltonzheng 3 years ago
Quentin-Anthony Quentin-Anthony requested a review from minjiaz minjiaz 3 years ago
Quentin-Anthony Quentin-Anthony requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 3 years ago
Quentin-Anthony Quentin-Anthony changed the title Add Unidirectional Sparse Attention Type to BigBird Add Unidirectional Sparse Attention Type to BigBird and BSLongformer 3 years ago
jeffra
jeffra approved these changes on 2022-05-20
jeffra Merge branch 'master' into qanthony/bigbird
a9c94211
jeffra jeffra enabled auto-merge (squash) 3 years ago
jeffra jeffra merged 5208eb73 into master 3 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum