PR #24631 Reduce llama-quantize peak memory use by 2.34x

Reduce llama-quantize peak memory use by 2.34x #24631

i386 wants to merge 7 commits into ggml-org:master from i386:codex/llama-quantize-low-memory

Reduce llama-quantize BF16 memory residency

b665f477

Release mmap tensor pages after quantization

f8305b74

i386 requested a review from

ggerganov 25 days ago

i386 changed the title ~~Reduce llama-quantize peak memory use~~ Reduce llama-quantize peak memory use by 2.34x 25 days ago

i386 closed this 25 days ago

i386 deleted the codex/llama-quantize-low-memory branch 25 days ago

i386 restored the head branch 25 days ago

i386 reopened this 25 days ago

Extract tensor float conversion helper

1814549e

Use type traits for tensor float conversion

99f2e63f

i386 marked this pull request as draft 25 days ago

Call tensor to_float directly

9f347c90

Keep tensor chunk offset assertions signed

4e98e7b7

Simplify stream chunk row calculation

a75f54dc

i386 marked this pull request as ready for review 25 days ago

ngxson closed this 22 days ago

Reviewers

ggerganov

Assignees

No one assigned

Labels

None yet

Milestone

No milestone