onnxruntime
7201dbeb - [DML EP] Split fused kernels when the persistent resource is too big (#16780)

Commit
2 years ago
[DML EP] Split fused kernels when the persistent resource is too big (#16780) The approach is the following: 1. Build partitions 2. Try compiling each partition into a `IDMLCompiledOperator` 3. If the compiled operator's persistent resource is bigger than 4GB, tell the partitioner to split the partition in the middle and try again. 4. Once all partitions have been successfully compiled into an `IDMLCompiledOperator`, fuse the partitions into an ORT operator and register them all. This change is relatively simple (basically a basic retry mechanism), but it required a lot of refactoring just to make sure that we don't modify the graph until **all** partitions have been compiled successfully. This is because partly modifying the graph before making sure that all partitions can be compiled will break future retries. This path is not expected to be used a lot, and even then the loop is not expected to loop more than twice very often. This is a very specific edge case for large models that were able to merge a large number of nodes into a single partition.
Parents
Loading