DeepSpeed
64c0052f - Ulysses HF Accelerate integration (#7638)

Commit
88 days ago
Ulysses HF Accelerate integration (#7638) Ulysses/ALST integration with HF Accelerate: - Allow `UlyssesSPAttentionHF.register_with_transformers` to get a `model` obj as an argument, to match HF accelerate's workflow - Fix existing Ulysses' tests to tests z2 instead of z1 - Improve documentation - Add a defensive check The HF Accelerate PR that depends on this PR is here https://github.com/huggingface/accelerate/pull/3817 --------- Signed-off-by: Stas Bekman <stas@stason.org>
Author
Parents
Loading