onnxruntime
ca3e9482 - Validate PackedMultiHeadAttention token offsets (#33098)

Commit
20 days ago
Validate PackedMultiHeadAttention token offsets (#33098) This pull request introduces validation for the `token_offset` input in the unfused `PackedMultiHeadAttention` CUDA operator, ensuring that all token offsets are within valid bounds. It adds a CUDA kernel for validation, integrates the check into the operator's logic, updates the operator schema documentation, and adds unit tests to verify the new validation logic. **Token Offset Validation for Unfused PackedMultiHeadAttention:** * Added a CUDA kernel (`ValidateTokenOffsetKernel`) and a new function (`ValidatePackedMultiHeadAttentionTokenOffset`) to validate that all `token_offset` values are within the range `[0, batch_size * sequence_length)`. If any value is out of bounds, the function returns an error. (`onnxruntime/contrib_ops/cuda/bert/packed_multihead_attention_impl.cu` [[1]](diffhunk://#diff-9bea87ca89683be050b0ae0961002626bfae523bea1c01acf0548e0389eeb86fR35-R81) `onnxruntime/contrib_ops/cuda/bert/packed_multihead_attention_impl.h` [[2]](diffhunk://#diff-0eb9ea035b0eac06d711eb187fbaf2ccf3d4165e5f25ab90db2b515b476f8991R16-R21) * Integrated the validation function into the unfused backend path of the `PackedMultiHeadAttention` operator, ensuring the check is performed before computation proceeds. (`onnxruntime/contrib_ops/cuda/bert/packed_multihead_attention.cc` [onnxruntime/contrib_ops/cuda/bert/packed_multihead_attention.ccR226-R235](diffhunk://#diff-0941dabf2a66e4f3e090c3c3562e2a493cf5daf905bfa1bd398f513f22ec708eR226-R235)) **Operator Documentation:** * Updated the operator schema documentation to specify that `token_offset` values must be in `[0, batch_size * sequence_length)`. (`onnxruntime/core/graph/contrib_ops/bert_defs.cc` [onnxruntime/core/graph/contrib_ops/bert_defs.ccL1024-R1025](diffhunk://#diff-8bf31275168b1e4a2aecd6760acf0ef92347134b003dfdc687c5d3cec4a178ecL1024-R1025)) **Testing:** * Added new unit tests to verify that invalid `token_offset` values in the unfused CUDA implementation are correctly detected and produce the expected error. (`onnxruntime/test/contrib_ops/packed_multihead_attention_op_test.cc` [onnxruntime/test/contrib_ops/packed_multihead_attention_op_test.ccR409-R462](diffhunk://#diff-c9634e9e2e9f2e69fd8c37c98888127f9417f69f340e8dbc5dd21815328be59bR409-R462))
Author
Parents
Loading