transformers
06e8101b - Merge branch 'cb-cache-layout' into cb-mla

Commit

5 days ago

Merge branch 'cb-cache-layout' into cb-mla

References

#50365 - Run multi-head latent attention models under continuous batching

#50369 - Decode multi-head latent attention from the cache pages with FlashMLA

Author

qgallouedec

qgallouedec

Parents

Loading