onnxruntime
be56d77a - Fix integer overflow in cuda NonMaxSuppression implementation (#2540)

Commit

6 years ago

Fix integer overflow in cuda NonMaxSuppression implementation (#2540) * add test case that should pass but fail * fix nms * extract int_max_output_boxes_per_class

References

#2540 - Fix integer overflow in cuda NonMaxSuppression implementation

Author

fs-eire

Parents

3e7aaf8f

onnxruntime be56d77a - Fix integer overflow in cuda NonMaxSuppression implementation (#2540)

onnxruntime
be56d77a - Fix integer overflow in cuda NonMaxSuppression implementation (#2540)