onnxruntime
661940ae - Account for MatMulNBits prepack memory

Commit
47 days ago
Account for MatMulNBits prepack memory Separate runtime workspace, persistent prepack buffers, and temporary prepack scratch in Level-1 estimates, and conservatively charge each component during CUDA partitioning. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Author
Parents
Loading