llama.cpp
Implement '--keep-split' to quantize model into several shards
#6688

Merged

Implement '--keep-split' to quantize model into several shards #6688

ggerganov merged 6 commits into ggml-org:master from zj040045:jiez/quantize-keep-split

Implement '--keep-split' to quantize model into several shards

17519e11

phymbert added split

Add test script

79bbf424

phymbert requested a review from

ggerganov 1 year ago

phymbert commented on 2024-04-18

ggerganov commented on 2024-04-19

Update examples/quantize/quantize.cpp

6d66e609

Split model correctly even if tensor id is out-of-order

d6e453eb

Update llama_model_quantize_params

141eb510

Fix preci failures

e0a3679a

ggerganov approved these changes on 2024-04-25

ggerganov merged 1966eb26 into master 1 year ago

Reviewers

ggerganov

phymbert

Assignees

No one assigned

Labels

split

Milestone

No milestone