DeepSpeed
d33b5623 - [Ulysses-ALST] add FA3 support (#7430)

Commit
146 days ago
[Ulysses-ALST] add FA3 support (#7430) FA3 is needed for 500K+ seqlen on llama-8b. Signed-off-by: Stas Bekman <stas.bekman@snowflake.com> Co-authored-by: Stas Bekman <stas.bekman@snowflake.com>
Author
Parents
Loading