vllm
b02fd288 - [Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. (#11787)

Commit

313 days ago

[Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. (#11787) Signed-off-by: Pavani Majety <pmajety@nvidia.com> Co-authored-by: mgoin <michael@neuralmagic.com>

References

#11787 - [Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models.

Author

pavanimajety

Parents

ff7424f4

vllm b02fd288 - [Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. (#11787)

vllm
b02fd288 - [Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. (#11787)