onnxruntime
1ea22669 - [DOCS ]Add annotated partitioning documentation (#27972)

Commit
112 days ago
[DOCS ]Add annotated partitioning documentation (#27972) This pull request introduces a new documentation page, `PartitioningWithAnnotationsAndMemoryConstraints.md`, which explains advanced ONNX Runtime features for partitioning model graphs across devices with explicit control. The doc covers how to annotate model layers for device assignment, collect per-node memory statistics, and enforce GPU memory budgets during partitioning. These features enable precise control over device placement and memory usage for large models. The most important changes are: **New Documentation: Advanced Partitioning Features** * Adds a comprehensive guide (`PartitioningWithAnnotationsAndMemoryConstraints.md`) describing how to use ONNX Runtime’s layer annotation and memory constraint features for graph partitioning. **Layer Assignment via Annotations** * Explains how to annotate ONNX model nodes with `layer_ann` metadata, including manual annotation and automated annotation using Olive’s `CaptureLayerAnnotations` pass. * Provides configuration examples for mapping annotation patterns to devices at runtime using the `session.layer_assignment_settings` session option. **Capacity-Aware Partitioning** * Details a two-phase workflow for profiling per-node memory usage and then enforcing a memory budget with the `session.resource_cuda_partitioning_settings` session option. * Covers both profiling-based and ad-hoc (estimation-only) approaches for memory-constrained partitioning. ([docs/annotated_partitioning/PartitioningWithAnnotationsAndMemoryConstraints.mdR1-R267](diffhunk://#diff-10b3051b9e36eccfc7ca0f2d44ce78a9980ca573cde0f931ffd1456da2c681daR1-R267) This is a follow up for https://github.com/microsoft/onnxruntime/pull/27595
Author
Parents
Loading