PR #28163 [Phi] Extend implementation to use GQA/MQA.

[Phi] Extend implementation to use GQA/MQA. #28163

ArthurZucker merged 15 commits into huggingface:main from phi-integration-update

chore(phi): Updates configuration_phi with missing keys.

e3d73a8e

gugarosa marked this pull request as draft 2 years ago

ArthurZucker commented on 2023-12-20

ArthurZucker commented on 2024-01-02

ArthurZucker commented on 2024-01-04

chore(phi): Adds first draft of combined modeling_phi.

3d3e9c30

gugarosa marked this pull request as ready for review 2 years ago

fix(phi): Fixes according to latest review.

8ed81465

fix(phi): Removes pad_vocab_size_multiple to prevent inconsistencies.

61bccf65

fix(phi): Fixes unit and integration tests.

89ccc4ff

fix(phi): Ensures that everything works with microsoft/phi-1 for firs…

f06c600d

fix(phi): Fixes output of docstring generation.

4322dd42

ArthurZucker approved these changes on 2024-01-08

fix(phi): Fixes according to latest review.

c8137ca3

fix(phi): Fixes according to latest review.

6b3055d1

Merge branch 'main' into phi-integration-update

11398a01

fix(tests): Re-enables Phi-1.5 test.

c46e0504

fix(phi): Fixes attention overflow on PhiAttention (for Phi-2).

f04fc40c

ArthurZucker commented on 2024-01-09

younesbelkada commented on 2024-01-10

fix(phi): Improves how queries and keys are upcast.

14fe45d7

Merge branch 'main' of github.com:gugarosa/transformers into phi-inte…

38d59b81

ArthurZucker commented on 2024-01-10

fix(phi): Small updates on latest changes.

ce2ebb40

ArthurZucker merged 55090585 into main 2 years ago

Reviewers

ArthurZucker

younesbelkada

susnato

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

transformers [Phi] Extend implementation to use GQA/MQA. #28163 Merged

[Phi] Extend implementation to use GQA/MQA. #28163

transformers
[Phi] Extend implementation to use GQA/MQA.
#28163

Merged