PR #36400 Add DeepSeek V2 Model into Transformers

Add DeepSeek V2 Model into Transformers #36400

Cyrilvallez merged 55 commits into huggingface:main from VladOS95-cyber:add-deepseekv2

VladOS95-cyber force pushed from a43e9e3d to 1a4f2d45 311 days ago

VladOS95-cyber force pushed from 1f95056a to 17ffdce1 302 days ago

VladOS95-cyber force pushed from be7eabf5 to c7c1d28a 299 days ago

VladOS95-cyber marked this pull request as ready for review 299 days ago

github-actions requested a review from

ArthurZucker 299 days ago

github-actions requested a review from

Rocketknight1 299 days ago

VladOS95-cyber force pushed from 5aeadaa7 to e2928965 297 days ago

VladOS95-cyber force pushed from beac6a57 to 99d5d6c2 296 days ago

VladOS95-cyber force pushed from 99d5d6c2 to a8ed8861 295 days ago

ArthurZucker commented on 2025-03-20

VladOS95-cyber force pushed from 7dae9da8 to f9a98e46 283 days ago

VladOS95-cyber requested a review from

ArthurZucker 283 days ago

VladOS95-cyber force pushed from 5c019739 to 7ae44890 282 days ago

VladOS95-cyber force pushed from 7ae44890 to 04522eee 281 days ago

add initial structure

e690c32f

doc fixes, add model base logic

4c4e6e43

update init files

f544bbad

some fixes to config and modular

841e47ac

some improvements for attention

436551b1

format

03e4b081

remove unused attn

42c7a067

some fixes for moe layer and for decoder

53c674af

adapt _compute_yarn_parameters for deepseek

a6dc5bb6

format

16a74e36

small fix

2c6771ff

fix for decoder forward

f19cd716

add tests, small refactoring

4a1fecea

fix dummies

221a8f41

fix init

5899ceec

fix doc

1ab69681

fix config docs

a59cd271

add sequce doc, fix init for gate

b8fe8aba

fix issues in tests

a2bfdb4c

fix config doc

a587a1f2

remove unused args

a6eea985

some fixes and refactoring after review

9d190553

fix doc for config

faf0a096

small fixes for config args

577165a5

revert config refactoring

5028c49d

small refactoring

073731a8

minor fixes after rebase

65ae28aa

VladOS95-cyber force pushed from 04522eee to 65ae28aa 280 days ago

Merge remote-tracking branch 'upstream/main' into add-deepseekv2

6b8b757d

small fix after merge

f7ad7bf8

fix modular

39d504a3

remove rotaryembd from public init

832d548b

small test fix

6ecbcb2f

ArthurZucker commented on 2025-05-26

Merge remote-tracking branch 'upstream/main' into add-deepseekv2

168a5968

some rotary pos calculation improvement

9af8f166

fix format

22aea3e6

VladOS95-cyber requested a review from

ArthurZucker 211 days ago

Merge remote-tracking branch 'upstream/main' into add-deepseekv2

1ff51da2

Cyrilvallez commented on 2025-06-16

Merge remote-tracking branch 'upstream/main' into add-deepseekv2

e126455a

some improvements and fixes

3a023fd7

fix config

5f1ff0a8

VladOS95-cyber requested a review from

Cyrilvallez 198 days ago

Cyrilvallez commented on 2025-06-25

Merge remote-tracking branch 'upstream/main' into add-deepseekv2

834650bc

some refactoring

8625b470

VladOS95-cyber requested a review from

Cyrilvallez 187 days ago

adjust some unit tests

60f6c4e4

skip test

335140bb

Cyrilvallez approved these changes on 2025-07-02

Merge branch 'main' into add-deepseekv2

773b4a94

Merge remote-tracking branch 'upstream/main' into add-deepseekv2

711a4455

small fixes and tests adjustment

40e28e14

Merge branch 'add-deepseekv2' of https://github.com/VladOS95-cyber/tr…

8f840578

VladOS95-cyber requested a review from

Cyrilvallez 183 days ago

Merge branch 'main' into add-deepseekv2

47918fb9

reapply modular

3fbb7e17

fix all tests except Integration

59e1ddb8

fix integration testzs

019efb51

cleanup BC stuff

8e859354

rope

89a2e717

fix integrations tests based on a10

4733bf17

style

797bf101

Cyrilvallez approved these changes on 2025-07-09

Cyrilvallez merged c9809042 into main 178 days ago

ArthurZucker added New model

Reviewers

Cyrilvallez

ArthurZucker

Rocketknight1

Assignees

No one assigned

Labels

New model

Milestone

No milestone

transformers Add DeepSeek V2 Model into Transformers #36400 Merged

Add DeepSeek V2 Model into Transformers #36400

transformers
Add DeepSeek V2 Model into Transformers
#36400

Merged