FEAT Add cartridges to PEFT (#2953)
Paper: https://arxiv.org/abs/2506.06266
Blog post:
https://hazyresearch.stanford.edu/blog/2025-06-08-cartridges
This method allows to learn a compressed KV cache with the help of a
corpus based on synthetic self-study dataset. The base implementation is
equivalent to prefix tuning.
The new initialization options work with prefix tuning too.
