transformers
0c98a994 - Migrate torchao integration off deleted torchao.dtypes (#47797)

Commit
109 days ago
Migrate torchao integration off deleted torchao.dtypes (#47797) torchao removed the torchao/dtypes package (AffineQuantizedTensor and the Layout classes). This updates the transformers torchao integration to stop importing from torchao.dtypes so it no longer crashes against current torchao, mirroring the approach in transformers #45321. Changes: - integrations/torchao.py: drop the `_quantization_type` helper and its `from torchao.dtypes import AffineQuantizedTensor` isinstance branch; `_linear_extra_repr` now detects quantized weights inline via `isinstance(self.weight, TorchAOBaseTensor)` (from torchao.utils). - tests/quantization/torchao_integration/test_torchao.py: replace the `from torchao.dtypes import AffineQuantizedTensor` import with `from torchao.utils import TorchAOBaseTensor` and update the isinstance assertions accordingly. - docs/source/en/quantization/torchao.md: remove the MarlinSparseLayout ("int4-weight-only-24sparse") examples and migrate the Int4CPULayout example to `Int4WeightOnlyConfig(int4_packing_format=PLAIN_INT32)`. Test plan: - `python -c "import transformers.integrations.torchao"` imports cleanly. - `RUN_SLOW=1 pytest tests/quantization/torchao_integration/test_torchao.py`: all 18 TorchAOBaseTensor isinstance / FqnToConfig tests pass. The 3 remaining failures (test_int4wo_quant, test_int4wo_offload, test_serialization_accelerator_*_Int4WeightOnlyConfig) are pre-existing generation-output text comparisons pinned to torchao int4 kernel numerics on the test hardware, unrelated to this change. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Author
Parents
Loading