onnxruntime
5d6cbb67 - [webgpu] Per-graph buffer manager for WebGPU multi-graph capture (#28260)

Commit
73 days ago
[webgpu] Per-graph buffer manager for WebGPU multi-graph capture (#28260) ### Summary Add per-annotation-ID buffer managers and captured command storage so multiple generators can each capture and replay their own graph independently without cross-contamination Add ReleaseGraph API through the full ORT stack (EP base → C API → InferenceSession → plugin EP) to release captured commands and GPU buffers when a generator is destroyed Replace the single graph_buffer_mgr_ / is_graph_captured_ bool with per_graph_buffer_mgrs_ map and captured_graph_ids_ set keyed by annotation ID Use a std::function getter with cached pointer pattern in GpuBufferAllocator to dynamically route allocations to the active per-graph buffer manager during runs, while keeping Alloc/Free as simple pointer dereferences ### Motivation Edge's Prompt API speed benchmark creates multiple sessions/generators sequentially with graph capture enabled. With the existing single-graph design, the second generator replays the first generator's captured commands with wrong buffers, producing incorrect output and ultimately a QuotaExceededError in the browser. This PR isolates each generator's graph capture state so they don't interfere with each other. ### Related PR The GenAI side change is in https://github.com/microsoft/onnxruntime-genai/pull/2106, which calls SessionReleaseGraph when a generator is destroyed to release the captured graph's GPU buffers. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: qjia7 <4221210+qjia7@users.noreply.github.com>
Author
Parents
Loading