transformers
Fine-grained quantization: a host quantizer that mixes methods per module
#49818
Open
Open
Fine-grained quantization: a host quantizer that mixes methods per module #49818
IlyasMoutawwakil
wants to merge
18 commits into
fg-split/3-experts-tp
from
fg-split/4a-host
IlyasMoutawwakil
changed the title 
IlyasMoutawwakil
changed the title
IlyasMoutawwakil
changed the title
Add a fine-grained quantizer that hosts one quantizer per module group
4a010e78
Write down what the fine-grained host needs from a member
cd471974
Tidy the host's docstrings and error messages
b0845301
Make the dtype_plan the one place that says which module gets which c…
6d73de37
Never skip the parents of a member's modules
54dbf044
Let a default skip yield to an entry that names the module
11af9414
Make the plan the only module selection on FineGrainedConfig
45b16848
Check only the None entry a lent skip-list adds
2f07960d
Quantize each tensor on the fly through its member's op
98359420
Let any quantizer that builds its skip-list with the base helper join…
88b0e8d5
Keep a conversion plain when its members agree and run only model ops
e8a6ce2f
Give each member its own conversions again after load
0e876cdf
Refuse a skip-list on FineGrainedConfig
46c83204
Scope members through their own skip-list field
212d13f8
Resolve the plan in one pass over each module's checkpoint names
a35cd714
Merge a passed config into the plan in FineGrainedConfig
2c6264c4
Pick a module's entry from the patterns that matched it
5441a389
Merge remote-tracking branch 'origin/fg-split/3-experts-tp' into fg-s…
1b841cf1
Login to write a write a comment.
Login via GitHub