PR #101 ALiBi Implementation

ALiBi Implementation #101

thomasw21 merged 7 commits into bigscience-workshop:main from ofirpress:main

alibi implementation

9aa94ab6

works but slow

47922189

efficient implementation

7ad5b482

ibeltagy requested a review from

thomasw21 4 years ago

ibeltagy requested a review from

TevenLeScao 4 years ago

ibeltagy requested a review from

slippylolo 4 years ago

ibeltagy requested a review from

ibeltagy 4 years ago

ibeltagy approved these changes on 2021-09-16

Update transformer.py

85c73f25

Update transformer.py

33503226

thomasw21 commented on 2021-09-16

Remove unnecessary .view()

06dc2d7f

bugfix with beta value in attention computation

5b5afb22

thomasw21 force pushed from a49461cd to 5b5afb22 4 years ago

TevenLeScao approved these changes on 2021-09-24

thomasw21 merged c839a8aa into main 4 years ago

Reviewers

TevenLeScao

ibeltagy

thomasw21

stas00

slippylolo

Assignees

No one assigned

Labels

None yet

Milestone

No milestone