PR #6942 Option to split during conversion

Option to split during conversion #6942

mofosyne merged 73 commits into ggml-org:master from christianazinn:convert-split

christianazinn marked this pull request as draft 2 years ago

support splits in convert.py

874c3411

christianazinn force pushed to 874c3411 2 years ago

Support split by size and dry run to write estimated shards/filesizes

72cbd4e0

Move split functionality to new GGUFManager class

702a7446

fix improper function signature

c33bdf39

tentative push of convert-hf-to-gguf support

b7c61208

mofosyne added Review Complexity : Medium

mofosyne added python

mofosyne added enhancement

Merge branch 'master' into convert-split

14b32915

resolve merge + SplitArguments for easier parsing

87a98a5b

mofosyne added help wanted

Merge remote-tracking branch 'origin' into convert-split

2dd78410

Fix eager tensor memory leak and remove convert.py changes

3ff27efa

refactor SplitStrategy to be a deque

6b5c3753

fix Q8 quantization

09baf2f3

remove unnecessary imports in gguf_manager

240243e6

Merge branch 'master' into convert-split

140eb52f

fix final? merge issue

a9c7703c

christianazinn marked this pull request as ready for review 2 years ago

github-actions added examples

compilade commented on 2024-06-03

fix gguf_writer placement and remove comments

efead040

oops, actually fix gguf_writer placement

c8ecbc67

compilade requested a review from

compilade 2 years ago

reduce duplicated code from gguf_writer

3e9430df

further simplify GGUFManager

f6fd3ea4

simplify even further and standardize with GGUFWriter

bb5ee020

reduce diffs with master

5ad397d6

form shards while adding tensors, SHA256 sums agree with master

ce7e6985

compilade commented on 2024-06-05

compilade commented on 2024-06-06

re-add type hint

706bd690

GGUFWriter compatibility fix

6a05183b

Shard dataclass and un-negative dont_add_architecture

3328b0a9

type consistency in format_n_bytes_to_str

1cbab222

move kv keys to constants.py

2037eabb

make pathlib explicit

83e4a3f5

base-1024 bytes to base-1000

13ffe22c

rename GGUFManager to GGUFWriterSplit

6d3a256d

compilade commented on 2024-06-07

Update gguf-py/gguf/constants.py

1312e287

fix convert-hf-to-gguf.py permissions

5f29d4a6

fix line endings

0283fc17

Update gguf-py/gguf/gguf_writer_split.py

dc5cf5fd

convert-hf : restore executable file permission

e093dfba

examples/convert-legacy-llama.py: restore executable file permission

9576965c

reinstate original gguf package import and fix type annotation

c6ae1d67

attempt to appease the linter

2e70fa10

attempt 2 to appease the linter

891b19cb

attempt 3 to appease the linter

02be0dd6

compilade commented on 2024-06-08

comma consistency

f658e91f

compilade commented on 2024-06-08

Update convert-hf-to-gguf.py

079dfe3a

edit cmd line args

282e71fb

Merge branch 'master' into convert-split

666bb097

use simplification from #7827

03cc9bcb

mofosyne marked this pull request as draft 2 years ago

kv/ti data are still wrong

97dd4169

try to refactor kv data (still fails)

ff2dd7d3

fix ti data messiness

ba1be979

Merge branch 'master' into convert-split

69d6e7a8

tidy up

0779f2f7

fix linting

a234bf82

actually make the linter happy

49b9fbe9

compilade commented on 2024-06-09

cleanup round 1

0471f67f

remove SplitStrategy, SplitArguments

5a96b8f2

appease linter

f7ecd996

compilade commented on 2024-06-09

fix typing and clean up

9d7f6944

fix linting

04171043

compilade commented on 2024-06-09