transformers
Decode multi-head latent attention from the cache pages with FlashMLA
#50369
Open
Overview
Commits
5
Changes
View On GitHub

Decode multi-head latent attention from the cache pages with FlashMLA #50369

qgallouedec wants to merge 5 commits into cb-mla from cb-mla-flashmla
qgallouedec
qgallouedec Decode multi-head latent attention from the cache pages with FlashMLA
7138261b
qgallouedec Merge branch 'cb-mla' into cb-mla-flashmla
9ebdd642
qgallouedec Describe the FlashMLA decode path in the architecture doc
9c27614b
qgallouedec Type the allocator that update_paged_rows reads pages from
79ec80f2
qgallouedec Batch prefills apart from decodes when latent attention decodes from …
7678a0f9
github-actions
HuggingFaceDocBuilderDev

Login to write a write a comment.

Login via GitHub

Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum