PR #613 Make loading weights 10-100x faster

Make loading weights 10-100x faster #613

jart merged 9 commits into ggml-org:master from jart:loader

Add mmap support for model files

2a6cef62

Fix ggml_init_params in quantize

a1e0f17a

Make mmap_file static

4ae12d08

Unmap the file in llama_free

4daaa5e7

Always initialize mm_addr and mm_length in llama_model

812cfa19

Initial windows support (untested)

80c2178d

jart added performance

jart added breaking change

jart force pushed to b806987b 2 years ago

jart force pushed from b806987b 2 years ago

bakkot commented on 2023-03-30

Make loading weights 10-100x faster

75d1e551

jart force pushed to 75d1e551 2 years ago

Ensure --mlock works properly with mmap() support

a45e843e

mqy commented on 2023-03-30

sw commented on 2023-03-30

jart force pushed 2 years ago

jart force pushed to adaba698 2 years ago

ggerganov approved these changes on 2023-03-30

Introduce GGML migration tool for new file format

516474b4

jart force pushed from adaba698 to 516474b4 2 years ago

jart merged ee0c40dd into master 2 years ago

Reviewers

ggerganov

pgoodman

Green-Sky

mqy

bakkot

Assignees

No one assigned

Labels

performance breaking change

Milestone

No milestone