PR #1588 Revamp medusa implementation so that every model can benefit.

Revamp medusa implementation so that every model can benefit. #1588

Narsil merged 16 commits into main from fix_medusa

[Tmp] Revamping medusa to make it orthogonal.

2446f3ec

Upgrade ALL the code.

ac419f5e

Small updates.

21b30722

Remove the old logic.

7a9998d4

Black.

64d38afa

Fix MPT, not sure about idefics.

f592df52

Fixing.

a0095b5b

Fix gemma + medusa.

ed95f198

Narsil changed the title ~~[WIP] Revamp medusa implementation so that every model can benefit.~~ Revamp medusa implementation so that every model can benefit. 2 years ago

OlivierDehaene commented on 2024-02-26

Fix GPT2 detection.

680a52f2

Download safetensors directly.

c7793235

Remove dead file.

1445b951

Specify revision to force use safetensors files.

fa40801f

Fix .

e672f976

Narsil force pushed from 52c8e22f to e672f976 2 years ago

OlivierDehaene dismissed these changes on 2024-02-26

Fixing revision for the medusa test.

bfec09ec

Narsil dismissed their stale review via bfec09ec 2 years ago

Forgot docker launcher.

915e5f08

Small fixes in the weights loading logic.

e69e68c8

Narsil merged bf700e7e into main 2 years ago

Narsil deleted the fix_medusa branch 2 years ago

Reviewers

OlivierDehaene

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

text-generation-inference Revamp medusa implementation so that every model can benefit. #1588 Merged

Revamp medusa implementation so that every model can benefit. #1588

text-generation-inference
Revamp medusa implementation so that every model can benefit.
#1588

Merged