llama.cpp
cuda : enable CUDA graphs for MMID 1 <= BS <= 4
#19645

Merged

cuda : enable CUDA graphs for MMID 1 <= BS <= 4 #19645

ggerganov merged 3 commits into master from gg/cuda-graphs-enable-bs-gt1

github-actions added Nvidia GPU

github-actions added ggml

ggerganov requested a review from

JohannesGaessler 4 days ago

ggerganov requested a review from

am17an 4 days ago

am17an commented on 2026-02-15

Base automatically changed from gg/graph-fix-kq-mask-reuse to master 3 days ago

ggerganov requested a review from

CISC 3 days ago

cuda : enable CUDA graphs for MMID BS <= 4

7d0be2c4

ggerganov force pushed from 15a68420 to 7d0be2c4 3 days ago

ORippler commented on 2026-02-16

ggerganov commented on 2026-02-16

cont : add stream capture check

dd9af011

JohannesGaessler approved these changes on 2026-02-17

cont : add MMVQ_MMID_MAX_BATCH_SIZE

573b94ca

ggerganov merged ad8207af into master 2 days ago

ggerganov deleted the gg/cuda-graphs-enable-bs-gt1 branch 2 days ago

Reviewers

JohannesGaessler

am17an

ORippler

CISC

Assignees

No one assigned

Labels

Nvidia GPU ggml

Milestone

No milestone