onnxruntime
36fc8c8a - [MIGraphX EP] Add migx ep fp8 support and int4 weights (#23534)

Commit
209 days ago
[MIGraphX EP] Add migx ep fp8 support and int4 weights (#23534) * Add fp8 and int4 types in supported list for Onnxruntime EP * Add support for int4 inputs Map things to int8 right now as we don't explicitly set an int4 input type and pack/unpack int4 operands * Add flag to allow for fp8 quantization through Onnxruntime API * Add fp8 quantization to the compile stage of the MIGraphX EP Mirror the same calibration code we use for int8 and just change which quantize we call through the MIGraphx API * cleanup logging * Cleanup and encapsulate quantization / compile functions - Add additional flags for fp8 thats shared for int8 - Add lockout warning message when int8/fp8 used at the same time * Run lintrunner pass * Fix session options inputs + add better logging. Previous runs using session options failed as we were missing pulling in inputs from the python interface. This plus additional logging allowed me to track what options were invoked via env and what were added during the start of an inference session * Fix naming for save/load path varibles to be consistent with enable. * Print only env variables that are set as warnings need this so the user knows there's any of the environment variables running in the background to ensure proper consistently between runs. --------- ### Description <!-- Describe your changes. --> Changes to cleanup the MIGraphX EP quantization code as well as adding fp8 quantization support along with int4 support. Cleanup changes handle a few instances of issues seen with the python interface when taking in provider options ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Required as we fix ignored flags when using provider_options for the MIGraphX EP Adding fp8 quantization through the MIGraphX API Adding int4 weight support for packed int4 weights for MIGraphX inference --------- Co-authored-by: Ted Themistokleous <tedthemistokleous@amd.com>
Parents
Loading