intel/auto-round

Pull Requests Commits

Implement AutoScheme support for model-free quantization; add validation and conversion functions, enhance documentation, and update tests for mixed-bit selection.

xin3he committed 20 hours ago

68df006c

Enhance documentation and comments for AutoScheme scoring parameters and methods; clarify default values for nsamples in environments.

xin3he committed 21 hours ago

1c5c35c4

Refactor _gen_layer_config to simplify nsamples and seqlen assignment for MoE models

xin3he committed 23 hours ago

56316a03

Add environment variables for AutoScheme nsamples and batch size; update documentation and tests

xin3he committed 23 hours ago

f2c6c230

revert nsamples change

xin3he committed 1 day ago

7b13fd4a

Enhance model_forward_low_gpu to support scheme tagging and improve logging of batch average loss

xin3he committed 1 day ago

0a78c646

set lm_head grad_mode=True

xin3he committed 1 day ago

115b5e49

add shared loss for inactived experts and refine log

xin3he committed 1 day ago

6b2d5a59

update log in autoscheme

xin3he committed 1 day ago

67df125c

fix bug of autoscheme

xin3he committed 1 day ago

6c564fba

fix auto_scheme gguf format wrong bits if use tiny model (#1953)

n1ck-guo committed 2 days ago

Verified b1a6baf9

speed up awq ci (#1954)

WeiweiZhang1 committed 5 days ago

Verified 9468e52c

add other non-integer scheme support for awq (#1892)

WeiweiZhang1 committed 6 days ago

Verified 5a3ade8c

Remove kernels version limit in test (#1945)

chensuyue committed 6 days ago

Verified 142e91b4

Use shared tokenizer save helper in exporters (#1951)

changwangss committed 7 days ago

Verified 436b9db6

update ARK README (#1922)

luoyu-intel committed 7 days ago

Verified f42a0421

Implement shard parallelism configuration and update documentation (#1942)

xin3he committed 7 days ago

Verified 7e2b69d4

fix saving hadamard rotation model without rotation config. (#1933)

lkk12014402 committed 7 days ago

Verified 6a2e5ad3

fix few gguf bugs of hf-name and checkpoint name (#1939)

n1ck-guo committed 8 days ago

Verified 126793d5

Add flashinfer-jit-cache installation to cuda unit test setup (#1947)

XuehaoSun committed 8 days ago

Verified 349b0418

fix issue of step3p7 and CI failure (#1929)

xin3he committed 9 days ago

Verified 1c4baa03

Update PyTorch and torchvision versions to 2.12.1 and 0.27.1 in CI (#1943)

XuehaoSun committed 9 days ago

Verified 354c6a3b

Update CPU agent pool (#1936)

chensuyue committed 9 days ago

Verified 35650a09

Fix SIGSEGV in pack-weight unpack/GEMM on malformed blob (#1928)

luoyu-intel committed 9 days ago

Verified 495e2539

Refine AutoScheme logic for gguf, reduce memory consumption and improve speed (#1916)

wenhuach21 committed 10 days ago

Verified 0c9c5b1d

DDP for multiple cards (#1897)

yiliu30 committed 11 days ago

Verified 33b7df0e

reorder UT for CUDA CI memory leak (#1938)

xin3he committed 13 days ago

Verified 0cc47c69

update what's new: vllm-omni integrates AutoROund (#1931)

xin3he committed 14 days ago

Verified 759a9ad7

Enhance b70 check (#1937)

yiliu30 committed 14 days ago

Verified e8709167

Support Wan2.2 S2V Quantization (#1924)

changwangss committed 14 days ago

Verified ca378799

Older