Skip to content

[GH-3357] Move shared GeoPackage reader into Spark common - #3361

Merged
jiayuasu merged 1 commit into
apache:masterfrom
jiayuasu:refactor/geopackage-common
Sep 14, 2026
Merged

[GH-3357] Move shared GeoPackage reader into Spark common#3361
jiayuasu merged 1 commit into
apache:masterfrom
jiayuasu:refactor/geopackage-common

Conversation

@jiayuasu

Copy link
Copy Markdown
Member

Did you read the Contributor Guide?

Yes, I have read the Contributor Rules and Contributor Development Guide.

Is this PR related to a ticket?

Related to #3357. This is a prerequisite cleanup for #3358; it does not implement list_layers or close the issue.

What changes were proposed in this PR?

Move the six identical GeoPackage reader classes from the Spark 3.5, 4.0, and 4.1 modules into spark/common, keeping their packages and implementations unchanged.

  • Keep one GeoPackage data-source registration in common.
  • Move the identical GeoPackageReaderTest suite into common and remove the version-specific copies.
  • Remove 2,106 duplicate source and test lines. No new abstractions, version shims, dependencies, or public APIs are introduced.

The common module is already built separately for each Spark/Scala version. Existing shared data sources use the same layout. #3358 can be rebased onto the shared implementation after this lands.

How was this patch tested?

  • Clean shaded builds passed for Spark 3.5.0 / Scala 2.12, Spark 4.0.0 / Scala 2.13, and Spark 4.1.1 / Scala 2.13. Each used mvn clean package -pl spark-shaded -am -DskipTests -Dmaven.javadoc.skip=true with the corresponding Spark and Scala profiles.
  • All 18 file-based GeoPackage tests passed on each version, both with common-module production classes and with the packaged shaded JAR. This covers vector and raster reads, GeoPackage metadata, null date/datetime fields, and the hidden _metadata column.
  • Verified that the version-specific JARs no longer contain the GeoPackage reader and that each shaded JAR has one GeoPackage registration.
  • Confirmed that all seven moved Scala files are byte-for-byte identical to the original copies. Commit hooks and git diff --check passed.
  • The existing MinIO/S3 integration test was not run locally because Docker was unavailable. It remains in the shared suite for CI.

Did this PR include necessary documentation updates?

No, this PR does not affect any public API so no need to change the documentation.

@jiayuasu jiayuasu added this to the sedona-2.0.0 milestone Sep 14, 2026
@jiayuasu
jiayuasu merged commit 78cdd67 into apache:master Sep 14, 2026
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant