transformers
[`distributed`] Context parallelism: padding masks, sliding windows and chunked attention with sdpa
#50516

Closed

[`distributed`] Context parallelism: padding masks, sliding windows and chunked attention with sdpa #50516

3outeille wants to merge 2 commits into cp-7-auto-pad from cp-8-masks

[distributed] Rebuild the attention mask of the whole sequence under …

4615fb03

[distributed] Test padding, sliding windows and chunked attention wit…

fb226b82

3outeille force-pushed the cp-7-auto-pad branch from 4249c6db to 043fd08c 1 day ago

3outeille force pushed from 4a6cf258 to fb226b82 1 day ago

3outeille closed this 15 hours ago

3outeille deleted the cp-8-masks branch 15 hours ago

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Milestone

No milestone