quantize: be able to override metadata by key #6321
ggerganov
merged
2 commits into
master
from
ik/quantize_with_kv_overrides
quantize: be able to override metadata by key
fc4c2a6f
minor : spacing
9c5fd6be
ggerganov
approved these changes
on 2024-03-26
sakura-umi1 year ago
Can this also be able to override string type metadata? This has not been supported yet.
Login to write a write a comment.
This can be useful, e.g., for low-bit quantization where the experience has shown that one can improve the model by changing the
f_rms_normparameter. Instead of having to specify the metadata override each time the model is used, with this PR one can encode the override during quantization usingThe
--override-kvargument can be repeated multiple times.