transformers
a67a2685 - Train adapters on top of FP8 weights (#50031)

Commit
17 days ago
Train adapters on top of FP8 weights (#50031) * Pass the input gradient through FP8 layers, so adapters can train on top of them * Same backward for FP8GroupedLinear, with a contiguous gradient for the grouped matmul * Dequantize in place in the FP8 backward * Fit the EP sentinel comment in five lines --------- Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co> Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
Author
Parents
Loading