PR #10765 [Misc][Quark] Upstream Quark format to VLLM

[AMD] support Quark quantized format

kewang-xlnx committed 1 year ago

[Model] add param_name remap in experts in dbrx model

kewang-xlnx committed 1 year ago

[Model] support quark kv cache format in dbrx

kewang-xlnx committed 1 year ago

add kv cache remap method for quark format

kewang-xlnx committed 1 year ago

[AMD][Quark] Fix fails in pr checks

kewang-xlnx committed 1 year ago

[AMD][Quark] replace get_compressed_tensors_cache_scale with get_cache_scale in aria.py

kewang-amd committed 1 year ago

[AMD][Quark] remove quark dependency

kewang-xlnx committed 1 year ago

[AMD][Quark] fix mypy error

kewang-xlnx committed 1 year ago

auto fix yapf

kewang-xlnx committed 1 year ago

fix mypy error

kewang-xlnx committed 1 year ago

fix code format

kewang-xlnx committed 1 year ago

Update vllm/model_executor/layers/quantization/compressed_tensors/triton_scaled_mm.py

kewang-xlnx committed 1 year ago

Modify based on PR comments

kewang-xlnx committed 1 year ago

fix mypy error

kewang-xlnx committed 1 year ago

delete comments of get_cache_scale in compressed_tensors.py

kewang-xlnx committed 1 year ago

fix mypy error

kewang-xlnx committed 1 year ago

change get_compressed_tensors_cache_scale to get_cache_scale in gemma2.py

kewang-xlnx committed 1 year ago

add CI test for quark format

kewang-xlnx committed 1 year ago

fix mypy error in CI tests

kewang-xlnx committed 1 year ago

be consistent with origin vllm

kewang-xlnx committed 1 year ago

vllm [Misc][Quark] Upstream Quark format to VLLM #10765 Merged