diff --git a/README.md b/README.md index b063fb1..85236ab 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ **Transformez** builds and applies vertical transformations across geodetic, tidal, and model-based height references, from local datums to global surfaces. -Transformez resolves your input and output references, computes the optimal geodetic pathway on the fly, and generates **spatially varying shift grids** — where regional models end, the transformation continues across open ocean and inland using global proxies, coastal blending, and meter-based inland decay. +Transformez resolves your input and output references, computes the optimal geodetic pathway on the fly, and generates **spatially varying shift grids**. Where regional models end, Transformez continues across open ocean and inland using global proxies, coastal blending, and meter-based inland decay. Transformez is part of the [Continuous DEMs Project](https://continuous-dems.readthedocs.io/), an ecosystem of tools for modern, continuous digital elevation model generation. Originally incubated within CUDEM, the engine has evolved into a standalone datum transformation suite. @@ -64,7 +64,7 @@ shift_array = transformez.generate_grid( ) ``` -> ⚠️ Shift grids are always added to your elevation data — the sign conventions are handled internally. See the [methodology guide](https://transformez.readthedocs.io/en/latest/user_guide/methodology.html#the-datum-shift-sign-conventions) for the physical intuition. +> ⚠️ Shift grids are always added to your elevation data; the sign conventions are handled internally. See the [methodology guide](https://transformez.readthedocs.io/en/latest/user_guide/methodology.html#the-datum-shift-sign-conventions) for the physical intuition. ## 📚 Documentation diff --git a/docs/source/_static/validation_stations_plot_chesapeake_bay.png b/docs/source/_static/validation_stations_plot_chesapeake_bay.png index ea94b1e..b0c517b 100644 Binary files a/docs/source/_static/validation_stations_plot_chesapeake_bay.png and b/docs/source/_static/validation_stations_plot_chesapeake_bay.png differ diff --git a/docs/source/_static/validation_stations_plot_tampa_bay_fl.png b/docs/source/_static/validation_stations_plot_tampa_bay_fl.png index 1c78fd4..393dccd 100644 Binary files a/docs/source/_static/validation_stations_plot_tampa_bay_fl.png and b/docs/source/_static/validation_stations_plot_tampa_bay_fl.png differ diff --git a/docs/source/_static/validation_vdatum_hist_astoria_or.png b/docs/source/_static/validation_vdatum_hist_astoria_or.png index 659af6f..eee2640 100644 Binary files a/docs/source/_static/validation_vdatum_hist_astoria_or.png and b/docs/source/_static/validation_vdatum_hist_astoria_or.png differ diff --git a/docs/source/_static/validation_vdatum_hist_channel_islands_ca.png b/docs/source/_static/validation_vdatum_hist_channel_islands_ca.png index 886aea4..af3a7b8 100644 Binary files a/docs/source/_static/validation_vdatum_hist_channel_islands_ca.png and b/docs/source/_static/validation_vdatum_hist_channel_islands_ca.png differ diff --git a/docs/source/_static/validation_vdatum_hist_chesapeake_bay.png b/docs/source/_static/validation_vdatum_hist_chesapeake_bay.png index d4b5781..8f19a09 100644 Binary files a/docs/source/_static/validation_vdatum_hist_chesapeake_bay.png and b/docs/source/_static/validation_vdatum_hist_chesapeake_bay.png differ diff --git a/docs/source/index.md b/docs/source/index.md index d225392..062f8f8 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -4,6 +4,8 @@ **Transformez** builds and applies vertical transformations across geodetic, tidal, and model-based height references, from local datums to global surfaces. +Transformez resolves your input and output references, computes the optimal geodetic pathway on the fly, and generates **spatially varying shift grids**. Where regional models end, Transformez continues across open ocean and inland using global proxies, coastal blending, and meter-based inland decay. + Transformez is part of the [Continuous DEMs Project](https://continuous-dems.readthedocs.io/), an ecosystem of tools for modern, continuous digital elevation model generation. ## Key Features @@ -99,7 +101,7 @@ out_file = transformez.transform_raster( ## Learn More -Read the [User Guide](user_guide/index.md) to install Transformez, generate your first shift grid, and understand how it works — from reference inputs and the CLI/Python API through the [geodetic methodology](user_guide/methodology.md) behind the dynamic hub-and-spoke routing, sign conventions, and coastal blending. +Read the [User Guide](user_guide/index.md) to install Transformez, generate your first shift grid, and understand how it works; from reference inputs and the CLI/Python API through the [geodetic methodology](user_guide/methodology.md) behind the dynamic hub-and-spoke routing, sign conventions, and coastal blending. ```{toctree} :maxdepth: 2 diff --git a/docs/source/user_guide/cli_usage.md b/docs/source/user_guide/cli_usage.md index f4166f4..a79abfa 100644 --- a/docs/source/user_guide/cli_usage.md +++ b/docs/source/user_guide/cli_usage.md @@ -1,6 +1,6 @@ # 💻 Command Line Interface -The `transformez` command line tool lets you generate or apply vertical transformation grids. Commands follow the 1.0 architecture: `build` for shift-grid generation, `shift` for raster transformation, plus inspection and planning commands to examine references and transformation paths before running them. +The `transformez` command line tool lets you generate or apply vertical transformation grids. Commands follow the 1.0 architecture: `build` for shift-grid generation, `shift` for product transformation, plus inspection and planning commands to examine references and transformation paths before running them. Commands are organized into three groups: **Execution** (`build`, `shift`, `prefetch`), **Discovery** (`list`, `info`, `plan`), and **External engines** (`htdp`, `vdatum` for installing and managing the NGS HTDP and NOAA VDatum engines). Legacy commands `grid` and `raster` remain available as deprecated aliases for `build` and `shift`. diff --git a/docs/source/user_guide/index.md b/docs/source/user_guide/index.md index 759692e..7bd3da9 100644 --- a/docs/source/user_guide/index.md +++ b/docs/source/user_guide/index.md @@ -1,15 +1,15 @@ # User Guide -This guide covers everything you need to install, use, and understand Transformez — from your first shift grid to the geodetic reasoning behind it. +This guide covers everything you need to install, use, and understand Transformez; from your first shift grid to the geodetic reasoning behind it. -Transformez transforms raster elevations between vertical datums — tidal datums, geoids, and ellipsoids — anywhere on Earth. Instead of a fixed lookup table, it resolves your input and output [references](references.md), plans the optimal geodetic pathway, and produces a **spatially varying shift grid** that you can generate from the [CLI](usage.md#command-line-interface), build through the Python API, or apply directly to a raster. +Transformez transforms raster elevations between vertical datums (tidal datums, geoids, and ellipsoids) anywhere on Earth. Transformez resolves your input and output [references](references.md), plans the optimal geodetic pathway, and produces a **spatially varying shift grid** that you can generate from the [CLI](usage.md#command-line-interface), build through the Python API, or apply directly to a raster. ## How to read this guide -- **New to Transformez?** Follow the guide in order — [Installation](installation.md) → [Usage](usage.md) → [References](references.md) → [Methodology](methodology.md). The whole path takes about fifteen minutes. +- **New to Transformez?** Follow the guide in order: [Installation](installation.md) → [Usage](usage.md) → [References](references.md) → [Methodology](methodology.md). - **Just want the commands?** Jump to [Usage](usage.md) for CLI and Python examples, or the [full CLI reference](cli_usage.md). - **Unsure what `-I vdatum:mllw` means?** See [Vertical References](references.md) for authority CRSs, namespaced surfaces, and the reference/binding model. -- **Applying a shift grid to your own data?** Read the [sign conventions](methodology.md#the-datum-shift-sign-conventions) first — the shift grid is always **added**, and the reasoning is not what most people expect. +- **Applying a shift grid to your own data?** Read the [sign conventions](methodology.md#the-datum-shift-sign-conventions) first: the generated shift grid is always **added**. - **Building flood or surge models?** The [inland decay](methodology.md#inland-tidal-decay) section covers when to disable coastal attenuation. - **Curious how a tidal-to-geoid pathway is actually built?** [Methodology](methodology.md) explains the hub-and-spoke routing, and the [VDatum Coverage Chains](vdatum_chains.md) appendix goes deep on mixed-generation VDatum handling. - **Need to trust the numbers?** [Validation & Accuracy](validation.md) shows measured agreement against NOAA CO-OPS, VDatum, FES/DTU, and NGS HTDP. @@ -17,6 +17,7 @@ Transformez transforms raster elevations between vertical datums — tidal datum > Using Transformez from another application? Transformez integrates with Fetchez and QGIS; see the [Integrations](usage.md#integrations) section. + ```{toctree} :maxdepth: 2 diff --git a/docs/source/user_guide/installation.md b/docs/source/user_guide/installation.md index daf41f7..61cc8f2 100644 --- a/docs/source/user_guide/installation.md +++ b/docs/source/user_guide/installation.md @@ -28,7 +28,7 @@ pip install git+https://github.com/continuous-dems/transformez.git#egg=transform ## External Engines -Transformez manages its external geodetic engines through the CLI, downloading and configuring them automatically. Neither engine is required for every transformation — Transformez will tell you when a requested pathway needs one. +Transformez manages its external geodetic engines through the CLI, downloading and configuring them automatically. Neither engine is required for every transformation: Transformez will tell you when a requested pathway needs one. ### HTDP @@ -44,7 +44,7 @@ By default HTDP installs into your user configuration. To install into the curre transformez htdp install --project ``` -If your workflows stay within static tidal/geoid transformations, HTDP may not be needed at all — Transformez will surface a clear message if a requested transformation requires it. +If your workflows stay within static tidal/geoid transformations, HTDP may not be needed at all. Transformez will surface a clear message if a requested transformation requires it. ### NOAA VDatum Java diff --git a/docs/source/user_guide/methodology.md b/docs/source/user_guide/methodology.md index 9b2b2d8..b45dbfe 100644 --- a/docs/source/user_guide/methodology.md +++ b/docs/source/user_guide/methodology.md @@ -1,5 +1,5 @@ # 📐 Geodetic Methodology & Architecture -To provide vertical transformations across varied geographic extents, Transformez relies on a dynamic, rigorous architecture. The engine computes optimal geodetic pathways on the fly — here's a look under the hood. +To provide vertical transformations across varied geographic extents, Transformez relies on a dynamic, rigorous architecture. The engine computes optimal geodetic pathways on the fly. Here's a look under the hood. ```mermaid flowchart LR @@ -18,9 +18,9 @@ The engine follows a typed reference → resolution → planning → execution p ## Reference Resolution -Before a transformation path is constructed, Transformez resolves user-supplied reference inputs into separate horizontal and vertical components. Standard CRS definitions are resolved through PROJ, while Transformez-specific tidal and model surfaces use explicit namespaced identifiers such as vdatum:mllw and global:lat. Legacy shorthand names are normalized to these references for backward compatibility. +Before a transformation path is constructed, Transformez resolves user-supplied reference inputs into separate horizontal and vertical components. Standard CRS definitions are resolved through PROJ, while Transformez-specific tidal and model surfaces use explicit namespaced identifiers such as `vdatum:mllw` and `global:lat`. Legacy shorthand names are normalized to these references for backward compatibility. -Reference resolution is separate from transformation execution: parsing determines what a reference represents, and the transformation engine determines how to connect the resolved source and destination through the appropriate geodetic models and hubs. This separation lets you inspect the full plan — providers, models, hubs, and frame transformations — before any data are fetched. +Reference resolution is separate from transformation execution: parsing determines what a reference represents, and the transformation engine determines how to connect the resolved source and destination through the appropriate geodetic models and hubs. This separation lets you inspect the full plan (providers, models, hubs, and frame transformations) before any data are fetched. ## The Dynamic Hub-and-Spoke Model @@ -61,7 +61,7 @@ flowchart TD | WGS84 | MLLW | WGS84 | Negative (↑ to ↓) | | LAT | MHHW | WGS84 | Depends on location | -* **Native Ellipsoid Hubs:** Every transformation is mathematically routed through a central geodetic frame (the "Hub"). +* **Native Ellipsoid Hubs:** Every transformation is routed through a central geodetic frame (the "Hub"). * **Intelligent Routing:** The engine evaluates the requested input and output datums and automatically selects the safest hub. For example, if both datums belong to the North American Datum family, the engine routes strictly through the NAD83 ellipsoid hub to avoid introducing unnecessary global transformation errors. If the request crosses international or global boundaries, it scales up to the WGS84 hub. @@ -72,7 +72,7 @@ A common point of confusion in vertical geodesy is the sign convention of shift * **The Stick in the Bay:** Imagine standing in the water of a bay holding a measuring stick with a "zero" line marked as Mean Low Water. If you move your "zero" mark to a higher datum (e.g., moving from Mean Low Water up to Mean Higher High Water), the water level on your stick will read as a lower number. -* **The Rule of Addition:** Because of this, shifting to a higher reference surface can require positive *or* negative shift values depending on location. Transformez automatically handles these complex sign inversions internally so you don't have to overthink it. You always simply **ADD** the generated shift grid to your raster (i.e., `New_DEM = Old_DEM + Shift_Grid`). The grid's native positive and negative values automatically ensure the math reflects physical reality. +* **The Rule of Addition:** Because of this, shifting to a higher reference surface can require positive *or* negative shift values depending on location. Transformez automatically handles these sign inversions internally so you don't have to overthink it. You always simply **ADD** the generated shift grid to your raster (i.e., `New_DEM = Old_DEM + Shift_Grid`). The grid's native positive and negative values automatically ensure the math reflects physical reality. * **Example:** @@ -100,7 +100,7 @@ Since water piles up and moves around and tides push into shallow bays and narro ## Continuous Coastal Blending -Official tidal models (like NOAA's VDatum) only provide data close to the coast. However, modern hydrodynamic modeling requires continuous grids that extend far into the deep ocean or miles inland. +Official tidal models (like NOAA's VDatum) provide data close to the coast. However, modern hydrodynamic modeling requires continuous grids that extend far into the deep ocean or miles inland. * **Offshore Extrapolation:** When a requested bounding box extends beyond native VDatum coverage, Transformez automatically fetches global satellite altimetry (like DTU25 or FES2014) as a proxy. diff --git a/docs/source/user_guide/providers.md b/docs/source/user_guide/providers.md index 85e6432..5058c38 100644 --- a/docs/source/user_guide/providers.md +++ b/docs/source/user_guide/providers.md @@ -11,4 +11,4 @@ Transformez automatically fetches model and reference data from a number of prov | Dist2Coast | Coastal context | internal | | HTDP | Frame/epoch | ellipsoidal frame changes | -Data fetching, retry, and caching are handled by Fetchez, Transformez's companion fetching library, so providers can be swapped or extended without touching the transformation engine itself. See [Methodology](methodology.md) for how these models are composed into a single transformation pathway. +Data fetching, retry, and caching are handled by [Fetchez](https://fetchez.readthedocs.io/en/latest/index.html), so providers can be swapped or extended without touching the transformation engine itself. See [Methodology](methodology.md) for how these models are composed into a single transformation pathway. diff --git a/docs/source/user_guide/references.md b/docs/source/user_guide/references.md index a2c8efe..4578ab6 100644 --- a/docs/source/user_guide/references.md +++ b/docs/source/user_guide/references.md @@ -1,6 +1,6 @@ # 🗺️ Vertical References -A reference describes the surface that elevations are expressed relative to. Transformez accepts two kinds of vertical input: +A reference describes the surface that elevations are expressed relative to. Transformez accepts multiple kinds of vertical input: | Form | Example | Meaning | | --------------------- | ---------------- | ---------------------------------- | @@ -32,7 +32,7 @@ Transformez conceptually separates vertical **references** from vertical **bindi - The **reference** describes *what* the surface is (e.g., NOAA's realization of MLLW). - The **binding** describes *how* Transformez realizes and operates on that surface. -Bindings encode the realization details — `provider`, `engine`, `provider-specific datum`, `native frame`, and `default model` — independently of the reference metadata. This separation lets the same reference evolve to new models or providers without changing user-facing input syntax. +Bindings encode the realization details, such as `provider`, `engine`, `provider-specific datum`, `native frame`, and `default model`, independently of the reference metadata. This separation lets the same reference evolve to new models or providers without changing user-facing input syntax. Not all references have a supported binding; see [Models and Providers](providers.md) for the current bindings. diff --git a/docs/source/user_guide/usage.md b/docs/source/user_guide/usage.md index 0d7ffa7..438aa2f 100644 --- a/docs/source/user_guide/usage.md +++ b/docs/source/user_guide/usage.md @@ -1,8 +1,8 @@ # 🛠️ Usage -Transformez can be used directly from the command line or Python API, or through integrations such as Fetchez and QGIS. All interfaces — CLI, Python API, and integrations such as Fetchez and QGIS — accept the same [references](references.md) and behave identically. +Transformez can be used directly from the command line or Python API, or through integrations such as Fetchez and QGIS. All interfaces (CLI, Python API, and integrations such as Fetchez and QGIS) accept the same [references](references.md) and behave identically. -> **🧮 Sign conventions:** vertical datum shifts are not intuitive — shifting to a *higher* surface does not simply mean positive values. Transformez handles all sign conventions internally. The one thing to remember: **always ADD the shift grid** to your elevation data. See the [sign conventions](methodology.md#the-datum-shift-sign-conventions) section of the methodology guide for the physical intuition. +> **🧮 Sign conventions:** vertical datum shifts are not always intuitive (shifting to a *higher* surface does not simply mean positive values). Transformez handles all sign conventions internally. The one thing to remember: **always ADD the shift grid** to your elevation data. See the [sign conventions](methodology.md#the-datum-shift-sign-conventions) section of the methodology guide for the physical intuition. ## Command Line Interface @@ -64,7 +64,7 @@ out_file = transformez.transform_raster( (which-interface-do-i-want)= ## Which interface do I want? -For most users, `transformez.generate_grid(...)` is all you need — it returns a 2D NumPy array of shift values, optionally saved to disk. +For most users, `transformez.generate_grid(...)` is all you need. This will return a 2D NumPy array of shift values, optionally saved to disk as a GIS-compatible raster. When you need a fully georeferenced, inspectable transformation product, use `build_shift_grid(...)` to obtain a `ShiftGrid` object. It carries the array, region, CRS, affine transform, source and target references, epochs, provenance, generation key, uncertainty, and cache identity, and can write or reproject itself. diff --git a/docs/source/user_guide/validation.md b/docs/source/user_guide/validation.md index 9a2799d..b7fb29f 100644 --- a/docs/source/user_guide/validation.md +++ b/docs/source/user_guide/validation.md @@ -11,11 +11,11 @@ The versions below record the Python packages and external geodetic engines reso | Component | Version | Source | | :--- | :--- | :--- | | **Transformez** | 1.0.0 | python environment | -| **Fetchez** | 0.8.7 | python environment | -| **HTDP** | 3.6.0 | resolved by Transformez | +| **Fetchez** | 0.8.8 | python environment | +| **HTDP** | 3.5.0 | resolved by Transformez | | **VDatum** | 4.8 | user | -> **Reproducibility note:** Transformez and Fetchez versions identify the Python implementation under test. HTDP and VDatum identify the external reference engines used by Tests 2 and 4; their resolved paths are recorded to make it explicit which managed or system installation was selected. +> **Reproducibility note:** Transformez and Fetchez versions identify the Python implementation under test. HTDP and VDatum identify the external reference engines used by Tests 2 and 4. ## Test 1: Production Coastal Surface vs. NOAA CO-OPS Tide Stations @@ -59,10 +59,10 @@ The Chesapeake Bay case exercises an even denser overlap environment, with numer | Region | VDatum Region | RMSE | Mean Difference | Points | Validation Challenge | | :--- | :--- | :--- | :--- | :--- | :--- | -| **Chesapeake Bay** | 5 | 0.023705 m | 0.008765 m | 84 | Estuary Shoaling | -| **Astoria OR** | 6 | 0.002611 m | 0.002592 m | 56 | River Dynamics | +| **Chesapeake Bay** | 5 | 0.022522 m | 0.004625 m | 84 | Estuary Shoaling | +| **Astoria OR** | 6 | 0.000385 m | -0.000062 m | 56 | River Dynamics | | **Tampa Bay FL** | 4 | 0.000365 m | -0.000049 m | 94 | Complex Bay Geometry | -| **Channel Islands CA** | 6 | 0.002810 m | 0.002795 m | 191 | Overlapping Legacy and Modern VDatum Coverage Chains | +| **Channel Islands CA** | 6 | 0.000326 m | -0.000002 m | 191 | Overlapping Legacy and Modern VDatum Coverage Chains | > **How to read this test:** This is a numerical implementation comparison, not a requirement for one-to-one reproduction of every internal VDatum software decision. Near-zero differences indicate that Transformez and VDatum evaluated effectively the same package and path. Larger localized differences, especially in dense overlap regions, should be interpreted in the context of package selection, mixed xGEOID/NAVD88 mosaicing, backend HTDP versions, grid interpolation, and each engine's overlap policy. The Channel Islands result is a regression check on mixed-generation package pairing and xGEOID/frame normalization; Chesapeake Bay additionally stresses multi-package overlap ordering. @@ -80,7 +80,7 @@ This is not an engine-equivalence test: the reference station values and the gri | Station | Published Offset | Transformez | Delta | | :--- | :--- | :--- | :--- | -![International Gauges](../_static/validation_international_bars.png) +![International Gauges](../_static/validation_international_bars.png)) > **How to read this test:** Agreement at the decimeter scale is meaningful here because the comparison is between a gridded global ocean model and local station realizations, not two implementations of the same transformation grid. The test is primarily a validation of global fallback selection and physical plausibility. @@ -93,7 +93,7 @@ These tests are best understood as integration or regression checks rather than | Test Region | Calculated Shift | Challenge | Status | | :--- | :--- | :--- | :--- | | **Washington (Cross-Epoch)** | -0.2610 m | Crustal Velocity & Datum Offset | PASS | -| **Japan (East Longitude)** | 1.9530 m | Eastern Hemisphere Longitude Parsing | PASS | +| **Japan (East Longitude)** | 1.9518 m | Eastern Hemisphere Longitude Parsing | PASS | > **How to read this test:** PASS indicates that the HTDP integration produced a plausible, finite result through the expected execution path. Detailed verification of HTDP's geophysical model belongs to NGS; these cases primarily protect Transformez against wrapper, frame-ID, epoch, and longitude-regression errors. diff --git a/docs/source/user_guide/vdatum_chains.md b/docs/source/user_guide/vdatum_chains.md index 86609eb..4d3b0ab 100644 --- a/docs/source/user_guide/vdatum_chains.md +++ b/docs/source/user_guide/vdatum_chains.md @@ -1,4 +1,4 @@ -# 🌊 VDatum Coverage Chains (Appendix) +# 🌊 VDatum Coverage Chains NOAA VDatum regional packages are treated by Transformez as coherent transformation units rather than as collections of interchangeable grids. diff --git a/tests/validation/validation.py b/tests/validation/validation.py index c3fbdc2..e037b2c 100644 --- a/tests/validation/validation.py +++ b/tests/validation/validation.py @@ -746,7 +746,7 @@ def generate_markdown_report( logger.info("Generating validation.md report...") md_lines = [ - "# Validation & Accuracy", + "# 🏹 Validation & Accuracy", "", "Transformez is validated at several different levels because no single benchmark can fully describe the behavior of a coastal vertical-datum transformation engine. The tests below separate provider/grid accuracy, production coastal behavior, global-model agreement, and external HTDP integration.", "", @@ -867,7 +867,9 @@ def generate_markdown_report( md_lines.extend( [ "", - "![International Gauges]({intl_stats['image']})" if intl_stats else "", + "![International Gauges](../_static/validation_international_bars.png))" + if intl_stats + else "", "", "> **How to read this test:** Agreement at the decimeter scale is meaningful here because the comparison is between a gridded global ocean model and local station realizations, not two implementations of the same transformation grid. The test is primarily a validation of global fallback selection and physical plausibility.", "",