llama.cpp
50e3e3e4 - CUDA: Remove redundant CUDA copies after SSM_SCAN (#29807)

Commit
28 days ago
CUDA: Remove redundant CUDA copies after SSM_SCAN (#29807) * CUDA: fuse copy of updated state snapshots into recurrent cache with ssm_scan * CUDA: remove redundant cuda copies with K==1 (non spec-dec) scenario as well
Author
Parents
Loading