text-generation-inference
Add cuda graphs sizes and make it default.
#1703
Merged

Add cuda graphs sizes and make it default. #1703

Narsil merged 3 commits into main from cuda_graph_default
Narsil
Narsil Move to cuda graphs by default (with possibility to choose graph sizes).
edcbc089
HuggingFaceDocBuilderDev
Narsil Clarify disabling.
6951962f
Narsil Fix disabling.
d67633a0
OlivierDehaene
OlivierDehaene approved these changes on 2024-04-04
Narsil Narsil merged 99874eae into main 1 year ago
Narsil Narsil deleted the cuda_graph_default branch 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
Milestone