transformers
pass module to Params4bit.from_prequantized to ensure quant_state
#32524

Merged

pass module to Params4bit.from_prequantized to ensure quant_state #32524

ArthurZucker merged 5 commits into huggingface:main from winglian:bnb-load-quant-state-fix

SunMarc commented on 2024-08-08

SunMarc requested a review from

Titus-von-Koeller 1 year ago

SunMarc requested a review from

matthewdouglas 1 year ago

matthewdouglas requested changes on 2024-08-08

matthewdouglas added Quantization

matthewdouglas added PyTorch FSDP

matthewdouglas approved these changes on 2024-08-08

SunMarc approved these changes on 2024-08-09

SunMarc requested a review from

ArthurZucker 1 year ago

pass module to Params4bit.from_prequantized to ensure quant_state

1c5d3c43

make sure to check bnb version

6e545501

revert min bnb version and use inspect on method instead

f5521cb4

use version instead of inspect to prevent performance hit

009ba7ca

make the property name readable

7a8c1f58

winglian force pushed to 7a8c1f58 1 year ago

muellerzr approved these changes on 2024-08-28

ArthurZucker approved these changes on 2024-08-29

ArthurZucker merged 39bfb2f5 into main 1 year ago

winglian deleted the bnb-load-quant-state-fix branch 1 year ago

Reviewers

ArthurZucker

matthewdouglas

SunMarc

muellerzr

Titus-von-Koeller

Assignees

No one assigned

Labels

Quantization PyTorch FSDP

Milestone

No milestone