[GH-3034] Spatial join planner: index ST_3DDWithin distance joins

[jiayuasu]

Did you read the Contributor Guide?

• Yes

Is this PR related to a ticket?

• Yes — closes #3034; follow-up to the Box3D EPIC (#2973).

What changes were proposed in this PR?

ST_3DDWithin was unplanned, so 3D distance joins ran as O(n × m) row-by-row evaluation. This PR wires both inferrable branches into the existing indexed distance-join pipeline.

Approach — same XY-projection trick as #3032 (Box3D ST_Intersects / ST_Contains), applied to the distance-join path. Correctness rests on the inequality |A_XY − B_XY|_2 ≤ |A − B|_3D, so expanding each XY rectangle by distance and probing the 2D R-tree is a valid superset filter; the actual 3D refine runs at the per-pair condition step.

• JoinQueryDetector: new arm for ST_3DDWithin(Seq(left, right, d)) that produces a JoinQueryDetection with SpatialPredicate.INTERSECTS for the distance-expanded envelope R-tree pass, condition (the full original join condition) as the per-pair filter, and distance = Some(d) so the executor builds the expanded envelope. Routes both overloads through the same plan — Geometry inputs land on the JTS envelope; Box3D inputs land on the XY footprint already materialised in TraitJoinQueryBase.shapeToGeometry by [GH-3031] Box3D spatial join: index ST_Intersects / ST_Contains on Box3D #3032. EXPLAIN output adds an ST_3DDWithin label.
• OptimizableJoinCondition: add ST_3DDWithin to the whitelist of distance-join predicates so isDistanceJoinOptimizable accepts it (parallel to the existing ST_DWithin entry).
• No changes to TraitJoinQueryBase required — Box3D's XY-footprint materialisation already lives there from [GH-3031] Box3D spatial join: index ST_Intersects / ST_Contains on Box3D #3032, and toExpandedEnvelopeRDD already calls into it.

ST_3DDWithin has no Geography overload, so no Geography branching is needed in the new arm (the analyzer would reject Geography input upstream).

How was this patch tested?

• New Box3DDWithinJoinSuite covers broadcast index distance join, non-broadcast DistanceJoinExec, the closed-interval threshold edge, and a discriminating row that is XY-overlapping with the left side but Z=99 away — the 2D R-tree pairs it via XY-expansion, the 3D refine rejects it. Also a Geometry-input case (POINT Z) that exercises the same indexed plan via the Geometry overload.
• Full sweep on Box3DDWithinJoinSuite + Box3DJoinSuite + Box3DDWithinSuite + Box2DJoinSuite: 30 tests pass locally.

Did this PR include necessary documentation updates?

• No — Box3D documentation is tracked separately under the Box3D EPIC.

[Copilot]

Pull request overview

This PR extends Sedona SQL’s spatial join planner so ST_3DDWithin joins (both Geometry and Box3D overloads) can use the existing indexed distance-join pipeline instead of falling back to row-by-row evaluation, improving join planning from O(n×m) toward indexed execution with a 2D superset filter + 3D refine.

Changes:

• Add ST_3DDWithin(left, right, distance) detection in JoinQueryDetector to plan an indexed distance join via 2D envelope expansion + per-pair refinement using the original predicate.
• Allow ST_3DDWithin in OptimizableJoinCondition so the optimizer accepts it as an eligible distance-join predicate.
• Add Box3DDWithinJoinSuite coverage for broadcast and non-broadcast indexed execution, threshold edge semantics, and correctness of the 3D refine vs XY-only candidates.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
spark/common/src/test/scala/org/apache/sedona/sql/Box3DDWithinJoinSuite.scala	Adds focused planner/executor tests ensuring ST_3DDWithin uses indexed join paths and that 3D refinement filters XY-only candidates correctly.
spark/common/src/main/scala/org/apache/spark/sql/sedona_sql/strategy/join/OptimizableJoinCondition.scala	Whitelists ST_3DDWithin as an optimizable distance-join predicate.
spark/common/src/main/scala/org/apache/spark/sql/sedona_sql/strategy/join/JoinQueryDetector.scala	Plans ST_3DDWithin using the distance-join pipeline (expanded 2D envelopes for indexing + original predicate for 3D refinement) and adds an EXPLAIN relationship label.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.