llama.cpp
39509fb0 - cuda : CUDA Graph Compute Function Refactor (precursor for performance improvements) (#11042)

Commit
218 days ago
cuda : CUDA Graph Compute Function Refactor (precursor for performance improvements) (#11042) * Refactor: Moves cuda graph executable update step to separate function. * Refactor: Moves cuda graph update check to separate function. * Refactor: Moves cuda graph maintenance (update or adjusting copy parameters) to separate function for improved readability. * Fix: Adds missing reference to maintain_cuda_graph() definition. * Refactor: Improves structure and abstractions by moving CUDA graph evaluation and capture to its own function. * Refactor: Moves node graph checks and copy ops into individual function for improved readability. * Refactor: Removes code permanently excluded from compilation to increase readability. * Style: Adds missing newline * Style: Consolidates several neighboring '#ifdef USE_CUDA_GRAPH' into a single one * Refactor: Makes 'cuda_graph_update_required' a local variable * remove double lines between functions --------- Co-authored-by: slaren <slarengh@gmail.com>
Author
Parents
  • ggml/src/ggml-cuda
    • File
      ggml-cuda.cu
Loading