All pull requests

Pull requests

• fix: stop struct outputs of the codegen dispatcher from leaking Arrow memory

  area:expressionsbug
  #6552·peterxcli opened Oct 2, 2026Member·

  ·

• docs: explain which Spark operators Comet leaves in place

  documentation
  #6551·andygrove opened Oct 2, 2026Member·Queued

• feat: support native wide-decimal hashing

  area:expressionsarea:shuffleenhancement
  #6548·rich7420 opened Oct 2, 2026Contributor·

  ·
  1

• perf: read the shuffle natively in operators AQE reuses from the initial plan

  enhancementperformance
  #6547·dwsmith1983 opened Oct 2, 2026Contributor·

  ·
  1

  1

• feat: [branch-1.1] reuse S3 credentials until shortly before their reported expiry (#6509)

  area:Icebergarea:scanenhancement
  #6545·andygrove opened Oct 2, 2026Member·

  ·

• fix: let a spilled final aggregate read its spill files back past its memory share

  area:aggregationarea:memorybugregression
  #6544·andygrove opened Oct 2, 2026Member·

  ·
  1

• chore: add benches remaining scalars

  area:expressions
  #6541·coderfender opened Oct 2, 2026Contributor·

  ·

• fix: keep Spark's cache format when Kryo would reject Comet's or Comet disables itself

  bug
  #6537·andygrove opened Oct 2, 2026Member·

  ·
  2

• refactor: move the remaining JNI-free entry point logic into core functions

  enhancement
  #6534·viirya opened Oct 2, 2026Member·

  ·

• Fix array_except type mismatch issue #3646

  area:expressions
  #6533·contentforge-press opened Oct 2, 2026·4 of 6 tasks·

  ·
  1

  6

• fix: fall back for Iceberg writes with float or double identity partitions

  area:Icebergarea:writerbug
  #6531·zhangfengcdt opened Oct 1, 2026Member·Queued
  1

  4

• docs: add release-behavior checks and the 1.1.0 audit's lessons to the review skills

  documentation
  #6516·andygrove opened Oct 1, 2026Member·

  ·
  2

• fix: read the Iceberg write gate's storage scheme the same way the native factory does

  area:Icebergarea:scanarea:writerbug
  #6502·0lai0 opened Oct 1, 2026Contributor·

  ·
  1

  3

• fix: prefer Unsupported when combining nested cast support

  area:expressionsbug
  #6501·liupoyi-1031 opened Oct 1, 2026Contributor·

  ·
  1

  2

• feat: Experimental local execution mode that runs a whole query as one DataFusion graph

  enhancementrun-spark-4.1-tests
  #6500·viirya opened Oct 1, 2026Member·

  ·
  1

  3

• fix: compact oversized nested list_extract buffers

  area:expressionsbug
  #6498·rich7420 opened Oct 1, 2026Contributor·

  ·
  1

  6

• perf: reduce Parquet runtime filter schema guard overhead

  area:joinsarea:scanenhancementperformancerun-all-spark-profiles
  #6496·pingzh opened Oct 1, 2026Contributor·

  ·
  1

  3

• test: share collation coverage across Spark 4.x

  enhancementrun-all-spark-profilestest
  #6495·sunchao opened Oct 1, 2026Member·

  ·
  1

• feat: per-location credentials for native Iceberg reads and writes

  area:Icebergarea:scanbackport-1.1enhancement
  #6478·snmvaughan opened Sep 30, 2026Contributor·

  ·
  1

  8

• fix: match Spark's float ordering for sort and window keys nested in arrays and structs

  area:expressionsbugregressionrun-spark-4.1-tests
  #6475·andygrove opened Sep 30, 2026Member·

  ·
  1

  12

• fix: Route collated array element comparisons through the codegen dispatcher

  area:expressionsbug
  #6471·viirya opened Sep 30, 2026Member·

  ·
  1

• docs: add release notes for 1.1.0 with known regressions and workarounds

  documentation
  #6469·andygrove opened Sep 30, 2026Member·

  ·
  1

• fix: coalesce shuffle partitions under Comet unions the way Spark does

  bug
  #6459·andygrove opened Sep 30, 2026Member·

  ·
  1

  9

• fix: match Spark's float ordering in min, max, greatest and least

  area:aggregationarea:expressionsbugrun-spark-4.1-tests
  #6457·andygrove opened Sep 30, 2026Member·

  ·
  11

• fix: normalize float operands in the native comparison builder

  area:expressionsbugrun-spark-4.1-tests
  #6447·andygrove opened Sep 30, 2026Member·

  ·
  1

  14