Skip to content

[GH-3357] GeoPandas: implement list_layers for GeoPackage - #3358

Open
jiayuasu wants to merge 1 commit into
apache:masterfrom
jiayuasu:feature/geopandas-list-layers
Open

[GH-3357] GeoPandas: implement list_layers for GeoPackage#3358
jiayuasu wants to merge 1 commit into
apache:masterfrom
jiayuasu:feature/geopandas-list-layers

Conversation

@jiayuasu

Copy link
Copy Markdown
Member

Did you read the Contributor Guide?

Yes, I have read the Contributor Rules and Contributor Development Guide.

Is this PR related to a ticket?

Closes #3357. Part of #2230.

What changes were proposed in this PR?

Add sedona.spark.geopandas.list_layers(filename) for a single GeoPackage. It returns a pandas DataFrame with name and geometry_type, sorted by name. Empty vector layers and registered nonspatial tables are included; raster tiles are excluded.

The implementation uses the existing Sedona GeoPackage reader, with no Pyogrio dependency or feature-geometry scan. A new includeGeometryType=true option extends showMetadata=true; the default ten-column schema and existing reader behavior remain unchanged. The opt-in mode rejects paths resolving to multiple files instead of silently using the first file.

Declared core geometry types include optional Z; M is not represented in the label, and generic geometry reports Unknown. Malformed feature metadata fails. JDBC resources and staged files are cleaned up, and remote staging avoids leftover Hadoop checksum sidecars.

This version supports .gpkg string/path-like inputs only. Remote inputs use Hadoop filesystem configuration and still require copying the entire file to executor-local temporary storage.

How was this patch tested?

  • Clean shaded-JAR builds for Spark 3.5, 4.0 and 4.1.
  • pytest -q tests/geopandas/test_io.py -k 'TestListLayers or read_geopackage': 34 passed each on PySpark 3.5.8, 4.0.2 and 4.1.1, using the freshly built JARs.
  • Native GeoPackageReaderTest metadata, vector, raster and hidden-metadata selections: 22 passed each on Spark 3.5.0, 4.0.0 and 4.1.1. Includes JDBC cleanup and staging through a real Hadoop ViewFileSystem mount.
  • Tests cover all 72 core-type/Z/M combinations, unreadable feature bytes, empty layers, nonspatial-only and tile-only catalogs, malformed metadata, projection, multiple files and invalid inputs. New Python tests run outside the Shapely-2-only class guard.
  • Confirmed the API/metadata tests fail without the implementation, and the checksum-sidecar regression fails against the previous reader.
  • Repository pre-commit checks and git diff --check passed.

All local runs used JDK 17. The Docker daemon is not running locally, so the MinIO/S3 test was not executed; its new metadata assertions and the remaining environment matrix are left to CI. The full repository suite was not run.

Did this PR include necessary documentation updates?

Yes. Added the API docstring for v2.0.0, GeoPackage reader documentation covering the option and limits, and a 2.0.0 release-note entry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GeoPandas: implement list_layers for GeoPackage

1 participant