PR #2479 feat: support phi3.5 moe

feat: support phi3.5 moe #2479

danieldk merged 15 commits into main from impl-phi-3-5-moe

drbh marked this pull request as ready for review 1 year ago

drbh changed the title ~~feat: support phi3.5 moe model loading~~ feat: support phi3.5 moe 1 year ago

drbh commented on 2024-09-03

danieldk commented on 2024-09-06

drbh force pushed from 4b8856d1 to a5fbbd19 1 year ago

danieldk requested a review from

danieldk 1 year ago

danieldk force pushed from 2133f826 to 41ce9329 1 year ago

danieldk force pushed from 41ce9329 to dc2c25bb 1 year ago

danieldk force pushed from dc2c25bb to b5fa8bde 1 year ago

Narsil dismissed these changes on 2024-09-27

feat: support phi3.5 moe model loading

853bc514

fix: prefer llama base model and improve rotary logic

dff1b9f7

feat: return reasonable generation and add integration test

1fb9d406

fix: run lint and update docs

1c917c03

fix: rerun lint for openapi docs

d3565552

fix: prefer do_sample false unless temp is set by user, and update ch…

b4e7601f

fix: small typo adjustments

b1026a84

fix: consolidate long rope paths

dad070b1

fix: revert greedy by default and test changes

c2c3e72e

Vendor configuration so that we don't have to `trust_remote_code`

f4cadd75

Use SparseMoELayer

c07c80fa

Add support for dense MoE

0cafbf3b

Some type annotations

245d6d8f

Add the usual model tests

e3e483c9

Ruff.

016cf4ec

danieldk dismissed their stale review via 016cf4ec 1 year ago

danieldk force pushed from 1921256f to 016cf4ec 1 year ago

Narsil approved these changes on 2024-09-30

danieldk merged 93a7042d into main 1 year ago

danieldk deleted the impl-phi-3-5-moe branch 1 year ago

Reviewers

Narsil

danieldk

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

text-generation-inference feat: support phi3.5 moe #2479 Merged

feat: support phi3.5 moe #2479

text-generation-inference
feat: support phi3.5 moe
#2479

Merged