transformers
8d87b3c4 - GGUF: fix MPS dequant byte-corruption and make norm de-offset data-driven

Commit
90 days ago
GGUF: fix MPS dequant byte-corruption and make norm de-offset data-driven Found via the slow GGUF suite on MPS. - dequantize_gguf_tensor now unwraps a GGUFQuantizedTensor input to a plain tensor up-front. Previously the dequant ops preserved the subclass, so the fp32 output still carried quant_type; the subclass's `.to` override then reinterpreted the storage as the target dtype during the loader's device cast on MPS (fp16/bf16 weights came out as raw-byte garbage like 28512.0; fp32 happened to survive). Fixes garbage generation on the MPS dequant path. - The Gemma/Nemotron `w + 1` norm de-offset still has to be pre-applied on the fp32 source (the loader casts to the target dtype before the converter chain runs — verified SubtractOne sees fp16, so it can't subtract safely). But it is no longer keyed off a hardcoded arch list: load_checkpoint_state derives the offset source patterns from the SubtractOne converters already present in weight_mapping, so any arch whose converter graph declares the offset works for free. SubtractOne is documented as the declarative marker it now is. Validated on MPS: gemma2/gemma3_text/nemotron weights-conversion and qwen2/stablelm generation tests pass; fp16/bf16/fp32 dequant all correct.
Author
Parents
Loading