PR #49818 Fine-grained quantization: a host quantizer that mixes methods per module

Add a fine-grained quantizer that hosts one quantizer per module group

IlyasMoutawwakil committed 12 days ago

Write down what the fine-grained host needs from a member

IlyasMoutawwakil committed 12 days ago

Tidy the host's docstrings and error messages

IlyasMoutawwakil committed 12 days ago

Make the dtype_plan the one place that says which module gets which config

IlyasMoutawwakil committed 12 days ago

Never skip the parents of a member's modules

IlyasMoutawwakil committed 12 days ago

Let a default skip yield to an entry that names the module

IlyasMoutawwakil committed 12 days ago

Make the plan the only module selection on FineGrainedConfig

IlyasMoutawwakil committed 12 days ago

Check only the None entry a lent skip-list adds

IlyasMoutawwakil committed 12 days ago

Quantize each tensor on the fly through its member's op

IlyasMoutawwakil committed 12 days ago

Let any quantizer that builds its skip-list with the base helper join a plan

IlyasMoutawwakil committed 12 days ago

Keep a conversion plain when its members agree and run only model ops

IlyasMoutawwakil committed 12 days ago

Give each member its own conversions again after load

IlyasMoutawwakil committed 12 days ago

Refuse a skip-list on FineGrainedConfig

IlyasMoutawwakil committed 12 days ago

Scope members through their own skip-list field

IlyasMoutawwakil committed 12 days ago

Resolve the plan in one pass over each module's checkpoint names

IlyasMoutawwakil committed 12 days ago

Merge a passed config into the plan in FineGrainedConfig

IlyasMoutawwakil committed 12 days ago

Pick a module's entry from the patterns that matched it

IlyasMoutawwakil committed 12 days ago

Merge remote-tracking branch 'origin/fg-split/3-experts-tp' into fg-split/4a-host

IlyasMoutawwakil committed 8 days ago

transformers Fine-grained quantization: a host quantizer that mixes methods per module #49818 Open