Generalize convert scripts #3838
Galunid
merged
43 commits into
ggml-org:master
from
Galunid:generic-convert
Initial generic convert script
4823b9bd
Remove comments
22201248
Make gguf_writer member of Model, rework tokenizer export
0ff23710
Add [UNTESTED] Baichuan support
8618b4e7
Missing variable
989db341
Missing variable
550b925a
cebtenzzre
commented
on 2023-10-29
Call add_tensor before write_* functions
443f7d58
MPT conversion fix
08918b70
Get rid of dumb print
3bb9844d
Add Falcon support
0afa75a9
Add Starcoder and Refact
94ba1db2
[Untested] Initial Persimmon support
6f6856c6
Woops
b9c664ab
Fix variable
0743f7a9
Add another alias to n_layers
dc3115f2
Flake8 fixes
b2ba44ea
Rework tokenizer handling
c94df097
Small refactor
235acc18
Remove 'old' conversion scripts
f4b9a7ea
Use 'IntEnum' instead of 'Enum'
3ec89dcc
Review fixes, persimmon fixes
4fdd7cdf
cebtenzzre
commented
on 2023-11-02
fix mypy errors
8f31dc54
sort imports
66ccd621
fix linter complaints
e9abcc9c
Green-Sky
commented
on 2023-11-02
model.py : add missing future import
007be850
Restore support for RWForCausalLM
03c9683e
Add big endian support
fd308505
Revert "Remove 'old' conversion scripts" - needed for testing
e64f4de1
Yarn rope for baichuan
2120195b
Rename convert-generic -> convert-hf-to-gguf
087f88cc
Move util to gguf-py/gguf
f7de892e
cebtenzzre
commented
on 2023-11-05
Move everything to convert-hf-to-gguf.py
781bc549
address review comments
fefc3db5
Fix flake8 complaints
648252ec
sort imports
05fb6f4e
store_true defaults to False, not None
7a3433b4
Change ftype from int value to str value
73780f59
Review fixes
88b0d9ef
Rename variable
b7148838
cebtenzzre
commented
on 2023-11-07
parse_args improvements
6a83bce1
Remove outdated scripts
2862d16f
Merge branch 'master' into generic-convert
c79c3585
Green-Sky
approved these changes
on 2023-11-08
Fix import path
2789ad93
ggerganov
approved these changes
on 2023-11-09
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub