transformers
Support passing flash_attn_kwargs when gradient_checkpointing is enabled
#37037

Merged

Support passing flash_attn_kwargs when gradient_checkpointing is enabled #37037

ArthurZucker merged 2 commits into huggingface:main from efsotr:gradient_checkpointing_flash_attn_kwargs

github-actions marked this pull request as draft 1 year ago

efsotr marked this pull request as ready for review 1 year ago

github-actions requested a review from

ArthurZucker 1 year ago

github-actions requested a review from

Rocketknight1 1 year ago

ArthurZucker approved these changes on 2025-03-27

support passing flash_attn_kwargs when gradient_checkpointing is enabled

379381ca

efsotr force pushed from 6ac945d9 to 379381ca 1 year ago

efsotr requested a review from

ArthurZucker 1 year ago

make modeling_deepspeek_v3.py consistent with modular_deepseek_v3.py

e3761eaa

ArthurZucker merged 2b4734bd into main 1 year ago

ArthurZucker added Flash Attention

Reviewers

ArthurZucker

Rocketknight1

Assignees

No one assigned

Labels

Flash Attention

Milestone

No milestone