Pull Requests apache/spark

[SPARK-53541][K8S][INFRA] Update K8s IT CI to use K8s 1.34 INFRA

#52293 opened 2025-09-09 22:29 by dongjoon-hyun

[SPARK-53029][FOLLOWUP] Update PyArrow StructType usage to avoid .names attribute for compatibility with PyArrow SQL PYTHON

#52289 opened 2025-09-09 18:31 by allisonwang-db

Feature/spark 47230 column prunning under explode SQL

#52288 opened 2025-09-09 15:38 by IgorBerman

[SPARK-48338][Core] Adding a Golden File Test With Randomly Generated SQL Scripts SQL

#52287 opened 2025-09-09 13:33 by TeodorDjelic

[SPARK-51489][SQL]Represent SQL scripts in Spark UI SQL WEB UI CORE

#52286 opened 2025-09-09 13:30 by dusantism-db

[SPARK-53532][BUILD][3.5] Upgrade Jetty to 9.4.58.v20250814 BUILD

#52283 opened 2025-09-09 07:15 by zml1206

[SPARK-53531][CORE] Better error message for HadoopRDD.getInputFormat CORE

#52282 opened 2025-09-09 06:38 by pan3793

[SPARK-53529][CONNECT] Fix spark connect client doesn't support ipv6 address SQL PYTHON CONNECT

#52280 opened 2025-09-09 03:06 by WangGuangxin

[SPARK-53530][SS] Clean up the useless code related to `TransformWithStateInPySparkStateServer` SQL STRUCTURED STREAMING PYTHON

#52279 opened 2025-09-09 02:59 by LuciferYang

[EXPERIMENT][DO NOT MERGE] Automated pyspark sdp build workflow CORE INFRA PYTHON

#52276 opened 2025-09-09 01:12 by AnishMahto

Catch InterruptedIOException and ClosedByInterruptException in TransformWithStateInPySparkStateServer to avoid worker crash SQL STRUCTURED STREAMING PYTHON

#52275 opened 2025-09-08 22:27 by jiateoh

Tiny doc wording clarity SQL DOCS PYTHON

#52273 opened 2025-09-08 17:53 by hutch3232

[SPARK-53525][CONNECT] Spark Connect ArrowBatch Result Chunking SQL PYTHON CONNECT

#52271 opened 2025-09-08 13:27 by xi-db

[SPARK-53524][CONNECT][SQL] Fix temporal value conversion in LiteralValueProtoConverter SQL CONNECT

#52270 opened 2025-09-08 13:17 by heyihong

[SPARK-53523][SQL] Named parameters respect `spark.sql.caseSensitive` SQL PYTHON

#52269 opened 2025-09-08 12:42 by pan3793

[SPARK-53516][SDP] Fix `spark.api.mode` arg process in SparkPipelines CORE

#52261 opened 2025-09-08 05:39 by pan3793

[SPARK-53448][PYTHON] Conversion of a pyspark DataFrame with a Variant column to pandas fails with an error SQL PYTHON

#52260 opened 2025-09-08 05:03 by VindhyaG

[SPARK-53512] [SQL] Better unification of DSv2 PushDownUtils SQL

#52257 opened 2025-09-07 22:43 by yhuang-db

[SPARK-53507]Add breaking change info to errors SQL CORE PYTHON CONNECT

#52256 opened 2025-09-05 23:43 by imarkowitz

[SPARK-53498][SDP] Correctly Reference `pyspark/pipelines/cli.py` from `spark-pipelines` Binary CORE PYTHON

#52254 opened 2025-09-05 20:54 by AnishMahto

[WIP][SPARK-53501][PYTHON][CORE][SQL][SS] refactor python worker factory API SQL STRUCTURED STREAMING CORE PYTHON

#52250 opened 2025-09-05 13:04 by benrobby

[MINOR][CORE][TESTS] Fix typo in MasterDecommissionSuite class name CORE

#52249 opened 2025-09-05 12:02 by ignitz

[SPARK-53355][PYTHON][SQL] fix numpy 1.x repr in type tests SQL PYTHON

#52247 opened 2025-09-05 08:59 by benrobby

feat: add indiamart_clone Flutter+Firebase starter app (dual login)

#52246 opened 2025-09-05 08:03 by pallasandeep-wq

[SPARK-53399][SQL] Collapse Python UDFs SQL PYTHON

#52238 opened 2025-09-04 16:06 by peter-toth

[SPARK-53491][SS] Fix exponential formatting of inputRowsPerSecond and processedRowsPerSecond in progress metrics JSON SQL STRUCTURED STREAMING

#52237 opened 2025-09-04 15:43 by jayantdb

[SPARK-53490][CONNECT][SQL] Fix Protobuf conversion in observed metrics SQL CONNECT

#52236 opened 2025-09-04 14:10 by heyihong

[SPARK-53492][CONNECT] Reject second ExecutePlan with an operation id that has already completed SQL CONNECT

#52232 opened 2025-09-04 09:29 by nija-at

[WIP][SPARK-52348][SDP] Trait for PipelinesHandler SQL CONNECT

#52228 opened 2025-09-04 04:23 by JiaqiWang18

[SPARK-53483][SQL] Support disable code-gen for sort merge join other than FULL OUTER and Existence SQL

#52227 opened 2025-09-04 03:21 by zml1206