llama.cpp
eeb5605d - CANN: Add MROPE and IMROPE support (#17401)

Commit
18 days ago
CANN: Add MROPE and IMROPE support (#17401) * CANN: ROPE supports both MROPE and IMROPE. 1. Optimize the caching logic of rope_cache_init. 2. Add support for mRoPE and i-mRoPE. Note that on Ascend 910B devices, it is necessary to disable FA in CLIP and disable NZ-format conversion. These two issues are still under investigation. * Resolve review comments
Author
Parents
Loading