diff --git a/docs/setup/emr.md b/docs/setup/emr.md index e39ac0ab8a5..873345fe12e 100644 --- a/docs/setup/emr.md +++ b/docs/setup/emr.md @@ -26,6 +26,8 @@ This tutorial is tested on EMR on EC2 with EMR Studio (notebooks). EMR on EC2 us ## Prepare initialization script +EMR already provides PySpark. The script below also installs packages for GeoPandas conversion and map visualization. See [optional Python dependencies](install-python.md#optional-dependencies) for the extras that provide these packages and how to avoid requesting another PySpark installation. + In your S3 bucket, add a script that has the following content: ```bash diff --git a/docs/setup/emr.zh.md b/docs/setup/emr.zh.md index a9adea4d41c..b0bda17d3b3 100644 --- a/docs/setup/emr.zh.md +++ b/docs/setup/emr.zh.md @@ -26,6 +26,8 @@ ## 准备初始化脚本 +EMR 已提供 PySpark。以下脚本还会安装用于 GeoPandas 转换和地图可视化的包。参见[可选 Python 依赖](install-python.md#optional-dependencies)，了解哪些附加依赖提供这些包，以及如何避免请求安装另一份 PySpark。 + 在您的 S3 存储桶中添加一个内容如下的脚本： ```bash diff --git a/docs/setup/install-python.md b/docs/setup/install-python.md index 6b780488017..1ea0e928ca0 100644 --- a/docs/setup/install-python.md +++ b/docs/setup/install-python.md @@ -17,13 +17,9 @@ under the License. --> -Apache Sedona extends pyspark functions which depends on libraries: +The base `apache-sedona` package installs `shapely` and `attrs`. Using Sedona with Spark also requires PySpark, which is often preinstalled on managed Spark platforms. -* pyspark -* shapely -* attrs - -You need to install necessary packages if your system does not have them installed. Sedona now uses [uv](https://docs.astral.sh/uv/) for Python dependency management. See the dependency definitions in our [pyproject.toml](https://github.com/apache/sedona/blob/master/python/pyproject.toml). +Use `pip` to install the published package and any extras you need. Sedona uses [uv](https://docs.astral.sh/uv/) to manage dependencies when developing the project. Package dependencies and version constraints are defined in [python/pyproject.toml](https://github.com/apache/sedona/blob/master/python/pyproject.toml). ### Install sedona @@ -36,7 +32,7 @@ pip install apache-sedona * Since Sedona v1.1.0, pyspark is an optional dependency of Sedona Python because spark comes pre-installed on many spark platforms. To install pyspark along with Sedona Python in one go, use the `spark` extra: ```bash -pip install apache-sedona[spark] +pip install "apache-sedona[spark]" ``` * Installing from Sedona Python source @@ -48,6 +44,38 @@ cd python python3 -m pip install . ``` +### Optional dependencies + +An extra installs additional packages for a particular use case. Choose the extras you need: + +| Package or extra | Additional packages | Use case | +| :--- | :--- | :--- | +| `apache-sedona` | `shapely`, `attrs` | Base package; use with an existing Spark installation. | +| `spark` | `pyspark` | Install PySpark when it is not already available. | +| `pydeck-map` | `geopandas`, `pydeck` | Create maps with `SedonaPyDeck`. | +| `kepler-map` | `geopandas`, `keplergl` | Create maps with `SedonaKepler`. | +| `flink` | `apache-flink` | Use Sedona with PyFlink. | +| `db` | `sedonadb[geopandas]` | Install SedonaDB with GeoPandas support; requested only on Python 3.9 or later. | +| `all` | `pyspark`, `geopandas`, `pydeck`, `keplergl`, `rasterio` | Install Spark, mapping, and Python raster dependencies together. | + +For example, to install the `all` extra: + +```bash +pip install "apache-sedona[all]" +``` + +Despite its name, `all` does not include the `flink` or `db` extra. Extras also do not install the Sedona JVM jars described below. + +The `rasterio` package supports Python-side raster objects. SQL readers for existing raster files do not require it. + +On a managed Spark platform such as EMR, keep the platform's PySpark installation. To add both mapping libraries without requesting PySpark, combine the mapping extras: + +```bash +pip install "apache-sedona[pydeck-map,kepler-map]" +``` + +GeoPandas brings in `pandas` as a dependency. PyArrow is also needed for Arrow-based conversions and pandas-on-Spark APIs; the `spark` and `all` extras do not explicitly request it. Install versions of `pandas` and `pyarrow` compatible with your Spark version when using these APIs. See [working with GeoPandas and Shapely](../tutorial/geopandas-shapely.md) for conversion examples. + ### Prepare sedona-spark jar Sedona Python needs one additional jar file called `sedona-spark-shaded` or `sedona-spark` to work properly. Please make sure you use the correct version for Spark and Scala. diff --git a/docs/setup/install-python.zh.md b/docs/setup/install-python.zh.md index c102d6f5a36..0365721f5a3 100644 --- a/docs/setup/install-python.zh.md +++ b/docs/setup/install-python.zh.md @@ -17,13 +17,9 @@ under the License. --> -Apache Sedona 扩展了 PySpark 函数，依赖以下库： +基础包 `apache-sedona` 会安装 `shapely` 和 `attrs`。在 Spark 上使用 Sedona 还需要 PySpark，托管 Spark 平台通常已预装该依赖。 -* pyspark -* shapely -* attrs - -如果您的系统尚未安装这些包，请先安装。Sedona 现使用 [uv](https://docs.astral.sh/uv/) 管理 Python 依赖，详细依赖定义见 [pyproject.toml](https://github.com/apache/sedona/blob/master/python/pyproject.toml)。 +使用 `pip` 安装已发布的包及所需的附加依赖。Sedona 项目开发使用 [uv](https://docs.astral.sh/uv/) 管理依赖。包依赖及版本约束见 [python/pyproject.toml](https://github.com/apache/sedona/blob/master/python/pyproject.toml)。 ### 安装 sedona @@ -36,7 +32,7 @@ pip install apache-sedona * 自 Sedona v1.1.0 起，pyspark 已成为 Sedona Python 的可选依赖，因为许多 Spark 平台都已预装 Spark。如需在安装 Sedona Python 时一并安装 pyspark，请使用 `spark` 附加项： ```bash -pip install apache-sedona[spark] +pip install "apache-sedona[spark]" ``` * 从 Sedona Python 源码安装 @@ -48,6 +44,38 @@ cd python python3 -m pip install . ``` +### 可选依赖 { #optional-dependencies } + +附加依赖（extra）会为特定用途安装额外的包，请按需选择： + +| 包或附加依赖 | 安装的额外包 | 用途 | +| :--- | :--- | :--- | +| `apache-sedona` | `shapely`、`attrs` | 基础包，可配合现有 Spark 环境使用。 | +| `spark` | `pyspark` | 在尚未安装 PySpark 时安装。 | +| `pydeck-map` | `geopandas`、`pydeck` | 使用 `SedonaPyDeck` 创建地图。 | +| `kepler-map` | `geopandas`、`keplergl` | 使用 `SedonaKepler` 创建地图。 | +| `flink` | `apache-flink` | 在 PyFlink 上使用 Sedona。 | +| `db` | `sedonadb[geopandas]` | 安装支持 GeoPandas 的 SedonaDB；仅在 Python 3.9 及以上版本请求安装。 | +| `all` | `pyspark`、`geopandas`、`pydeck`、`keplergl`、`rasterio` | 一并安装 Spark、地图和 Python 栅格相关依赖。 | + +例如，安装 `all` 附加依赖： + +```bash +pip install "apache-sedona[all]" +``` + +虽然名为 `all`，它并不包含 `flink` 或 `db` 附加依赖。附加依赖也不会安装下文所述的 Sedona JVM jar 文件。 + +`rasterio` 用于支持 Python 端的栅格对象。使用 SQL 读取现有栅格文件不需要它。 + +在 EMR 等托管 Spark 平台上，请保留平台提供的 PySpark。若要安装两个地图包而不请求安装 PySpark，可以组合使用地图附加依赖： + +```bash +pip install "apache-sedona[pydeck-map,kepler-map]" +``` + +GeoPandas 会通过依赖关系安装 `pandas`。基于 Arrow 的转换和 pandas-on-Spark API 还需要 PyArrow；`spark` 和 `all` 附加依赖不会显式请求安装它。使用这些 API 时，请安装与 Spark 版本兼容的 `pandas` 和 `pyarrow`。转换示例见[使用 GeoPandas 和 Shapely](../tutorial/geopandas-shapely.md)。 + ### 准备 sedona-spark jar Sedona Python 需要额外配套一个名为 `sedona-spark-shaded` 或 `sedona-spark` 的 jar 文件。请确保所选版本与您的 Spark 和 Scala 版本匹配。