vllm
[Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models.
#11787

Merged

[Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. #11787

simon-mo merged 6 commits into vllm-project:main from pavanimajety:modelopt-k-v-scales

mgoin requested a review from

mgoin 1 year ago

pavanimajety force pushed 1 year ago

pavanimajety marked this pull request as ready for review 1 year ago

mgoin commented on 2025-01-08

pavanimajety changed the title ~~[Hardware][NV] Fix Modelopt model loading for k-v-scales~~ [Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. 1 year ago

pavanimajety force pushed 1 year ago

[Hardware][NV] Fix Modelopt model loading for k-v-scales

16e4650e

Format

c02df146

NFC: remove print

aee253a5

Address Feedback

229ebe8e

Add scales to mixtral models as well

705cf4e6

pavanimajety force pushed to 705cf4e6 1 year ago

pavanimajety requested a review from

mgoin 1 year ago

Merge branch 'main' into modelopt-k-v-scales

045e2000

mgoin approved these changes on 2025-01-27

mgoin added quantization

mgoin added ready

mgoin enabled auto-merge (squash) 1 year ago

disabled auto-merge 1 year ago
Manually disabled by user

simon-mo merged b02fd288 into main 1 year ago

Reviewers

mgoin

Assignees

No one assigned

Labels

ready

Milestone

No milestone

vllm [Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. #11787 Merged

[Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. #11787

vllm
[Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models.
#11787

Merged