Cherry-picks into rel-1.21.0 (#23846)
Cherry-pick the following changes into
[rel-1.21.0](https://github.com/microsoft/onnxruntime/tree/rel-1.21.0).
- (#23791)
- (#23710)
- (#23789)
- (#23829)
---------
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
Co-authored-by: Yifan Li <109183385+yf711@users.noreply.github.com>
Co-authored-by: Ankit Maheshkar <ankit.maheshkar@intel.com>
Co-authored-by: n1harika <niharika.sathish@intel.com>
Co-authored-by: Changming Sun <chasun@microsoft.com>
