ecmwf · Oisin-M · Jul 30, 2026 · Jul 30, 2026 · Aug 18, 2026 · Aug 18, 2026
diff --git a/docs/development/contributing.rst b/docs/development/contributing.rst
@@ -0,0 +1,52 @@
+Contributing
+============
+
+**earthkit** is an open-source project, and contributions are highly welcomed and appreciated for all components.
+
+The code is hosted on GitHub:
+
+- `earthkit <https://github.com/ecmwf/earthkit>`_
+- `earthkit-data <https://github.com/ecmwf/earthkit-data>`_
+- `earthkit-plots <https://github.com/ecmwf/earthkit-plots>`_
+- `earthkit-meteo <https://github.com/ecmwf/earthkit-meteo>`_
+- `earthkit-geo <https://github.com/ecmwf/earthkit-geo>`_
+- `earthkit-transforms <https://github.com/ecmwf/earthkit-transforms>`_
+- `earthkit-hydro <https://github.com/ecmwf/earthkit-hydro>`_
+- `earthkit-utils <https://github.com/ecmwf/earthkit-utils>`_
+
+
+Development workflow
+--------------------
+
+1. Fork the repository on GitHub
+2. Clone the fork to your local machine
+3. Create a virtual environment and install the package in development mode
+4. Create a new branch for your changes
+5. Make your changes and commit them with a clear message
+6. Run tests to ensure everything is working correctly
+7. Push your changes to your fork on GitHub
+8. Open a pull request against the develop branch of the repository
+
+Code style
+----------
+This project uses ruff for code styling and formatting. To handle these automatically, you can use pre-commit hooks. To set them up, run:
+
+.. code-block:: bash
+
+    pip install pre-commit
+    pre-commit install
+
+Testing
+-------
+To run the tests, you can use pytest. Make sure you have all dependencies installed, then simply run:
+
+.. code-block:: bash
+
+    pytest
+
+Documentation
+-------------
+To contribute to the documentation, see the developer
+resources in the docs:
+
+- :doc:`documentation` — short guidance for authors (Diátaxis)
diff --git a/docs/development/docs.rst b/docs/development/docs.rst
diff --git a/docs/development/documentation.rst b/docs/development/documentation.rst
@@ -0,0 +1,88 @@
+Documentation Guidelines
+========================
+
+earthkit uses the Diátaxis framework, which organises documentation into four complementary types:
+
+- Tutorials — learning-oriented, step-by-step examples for newcomers.
+- How-to Guides — short recipes that solve a specific problem.
+- Concepts — conceptual background and rationale.
+- API Reference — factual API documentation and exhaustively-listed behaviour.
+
+In addition to these standard Diátaxis types, earthkit also two additional sections in the User Guide.
+
+- Installation and Getting Started - a quick way to get up and running.
+- Frequently Asked Questions - common questions with short answers
+
+Where to place content
+----------------------
+
+- Frequently Asked Questions: only for questions that often come up and have short answers (1-5 lines). If the answer is longer, it should be written elsewhere and linked to from the Frequently Asked Questions.
+
+- Tutorials: put hands-on, example-driven content.
+  Preface with a clear goal, required inputs, and a short worked example.
+
+- How-to Guides: add focused recipes`. Keep them
+  concise and task-oriented; show the recipe first, then explain options.
+
+- Concepts: add design rationale and conceptual material. These pages are for readers who want
+  "why" and "how it works" rather than "what to click".
+
+- API Reference: API docs should be
+  generated from docstrings as much as possible. Keep docstrings authoritative and minimal
+  narrative in the reference pages.
+
+Writing tips
+------------
+
+- Title your pages for intent (e.g. "Delineating catchments", not "Notes").
+- Start tutorials with "What you will learn" and a short, copy-pastable
+  example that runs quickly.
+- For how-to guides, lead with the exact commands or code that solves the
+  task; follow with explanation of options and common pitfalls.
+- Use short paragraphs and clear headings; aim for a single idea per
+  paragraph.
+- Prefer concrete examples over abstract descriptions in tutorials and
+  how-tos. Put conceptual material in Concepts pages.
+
+Writing examples
+----------------
+
+- Tutorial (first lines):
+
+  "This tutorial shows how to load a precomputed EFAS river network and
+  compute catchment statistics."
+
+- How-to (first lines):
+
+  "How to load a GRIB file as an xarray Dataset. Code:"
+
+  .. code-block:: python
+
+    import earthkit.data as ekd
+
+    data = ekd.from_source("file", "temperature.grib")
+    ds = data.to_xarray()
+
+- Concepts (first lines):
+
+  "What is a forecast step? Forecast data has both a reference time and a
+forecast step. The valid time is the sum of these two quantities. This
+page explains the relationship between reference time, step and valid
+time, and why they are represented separately."
+
+Keeping documentation high quality
+----------------------------------
+
+- Link to a single canonical location for each topic. Avoid duplicate
+  content across pages.
+- When adding API examples, keep them small and runnable and prefer
+  example snippets that do not require external datasets.
+- Submit documentation changes via pull requests and include a short
+  description that states what changed and why.
+
+For more detail, read the Diátaxis guide: https://diataxis.fr/
+
+Notebooks
+---------
+
+Ensure the title and the subchapter headings have the same size as in the other notebooks and are recognised and navigable in the documentation.
diff --git a/docs/development/guidelines.rst b/docs/development/guidelines.rst
diff --git a/docs/development/index.rst b/docs/development/index.rst
diff --git a/docs/development/principles/dependencies.rst b/docs/development/principles/dependencies.rst
@@ -0,0 +1,13 @@
+Dependencies
+============
+
+earthkit aims to be interoperable with a wide range of Python libraries.
+This means that adding hard dependencies for every supported data type will lead to a growing list of dependencies, despite the fact that most users will have no need for most of them.
+
+
+Each component should therefore keep its core dependencies to a minimum, depending only on libraries that are fundamental to its own functionality.
+In particular, NumPy should be considered a core dependency, while other third-party libraries should be added to a component's default dependency set only when they are required by its core functionality.
+Support for specific data formats, libraries, or integrations should generally be provided through optional dependencies.
+
+At the top-level earthkit package, however, we should favour a convenient out-of-the-box experience and include dependencies needed to support the most common use cases.
+Thus, while individual components should minimise their default dependencies, the top-level package may aggregate a broader set of optional functionality so that most users can install earthkit and have support for common use cases without installing additional dependencies.
diff --git a/docs/development/principles/functional-first.rst b/docs/development/principles/functional-first.rst
@@ -0,0 +1,30 @@
+Functional-first Design
+=======================
+
+earthkit follows a functional-first design approach. Functionality should
+primarily be expressed through functions operating on data, rather than through
+object-oriented hierarchies with complex inheritance structures.
+
+Prefer small, composable functions with clear inputs and outputs::
+
+    result = earthkit.foo.bar(data, options)
+
+over stateful objects that hide operations behind methods::
+
+    result = data.bar(options)
+
+Functions should:
+
+* have explicit inputs and outputs i.e. avoid `**kwargs`
+* avoid unnecessary mutable state
+* be easy to compose with other functions
+* work naturally with different supported data types
+
+Object-oriented patterns may still be used where they provide a clear benefit,
+for example for representing stateful resources, configuration, or complex
+lifecycle management. However, new APIs should default to a functional design
+unless there is a strong reason to introduce an object abstraction.
+
+A functional design also supports interoperability by allowing the same
+operation to be dispatched across different data backends while keeping the
+public API consistent.
diff --git a/docs/development/principles/index.rst b/docs/development/principles/index.rst
@@ -0,0 +1,18 @@
+Principles
+==========
+
+General principles/guidelines for developers to contribute to ECMWF software are publicly visible in our
+`Codex <https://github.com/ecmwf/codex/blob/main/Guidelines/External-Contributions.md>`_.
+
+earthkit-specific principles are also laid out here and take precedence.
+
+.. toctree::
+   :maxdepth: 2
+
+   maximum-interoperability
+   one-ground-truth
+   naming
+   which-package
+   functional-first
+   dependencies
+   missing-values
diff --git a/docs/development/principles/maximum-interoperability.rst b/docs/development/principles/maximum-interoperability.rst
@@ -0,0 +1,71 @@
+Maximum Interoperability
+========================
+
+earthkit is not an attempt to reinvent the wheel and retaining interoperability is a key goal. This means both interoperability between earthkit packages, and interoperability with the rest of the Python ecosystem.
+
+1. Interoperability with the Python ecosystem
+---------------------------------------------
+
+earthkit aims to integrate naturally with the wider Scientific Python ecosystem. The primary supported data types are:
+
+- xarray
+- Array API-compatible arrays (e.g. NumPy, CuPy, PyTorch)
+- pandas
+- earthkit-data objects
+
+The public function should contain only dispatch logic. Each backend should
+implement the same function with the same signature and semantics.
+
+A minimal made-up example implementing MSE as a function earthkit.foo.bar with xarray and array implementations::
+
+    # earthkit.foo
+
+    from earthkit.utils.decorators import dispatch
+
+    def bar(a, b):
+        """
+        Doc for toplevel implementation.
+        Links to backend implementations.
+        """
+        dispatched_function = dispatch(bar, fieldlist=False)
+        return dispatched_function(a, b)
+
+The backend implementations live in the corresponding submodules::
+
+    # earthkit.foo.array
+
+    from earthkit.utils.array import array_namespace
+
+    def bar(a, b):
+        """
+        Doc for array implementation.
+        """
+        xp = array_namespace(a, b)
+        # array-api compat logic
+        return xp.vector_norm((a-b), ord=2)
+
+
+
+    # earthkit.foo.xarray
+
+    def bar(a, b):
+        """
+        Doc for xarray implementation.
+        """
+        # xarray logic
+        return ((a-b)**2).mean(skipna=False)
+
+This structure keeps the public API independent of the supported input types,
+avoids unnecessary data conversion, and makes it straightforward to add support
+for additional backends. Read :doc:`one-ground-truth` for guidance regarding how to
+handle multiple implementations with minimal maintenance burden e.g. xarray and array
+above.
+
+.. important::
+
+    Dispatching and array-api compat both rely on being able to detect the desired backend from inputs. This is not always possible. Numpy is preferred when array-api compat is infeasible, and xarray is preferred for the toplevel function when dispatching is infeasible.
+
+2. Interoperability between earthkit packages
+---------------------------------------------
+
+earthkit should work seamlessly as an ecosystem and therefore packages should be easily interoperable between each other by supporting the same data formats, APIs, naming etc. as much as possible.
diff --git a/docs/development/principles/missing-values.rst b/docs/development/principles/missing-values.rst
@@ -0,0 +1,14 @@
+Missing Value Handling
+======================
+
+The default policy for missing values should be to propagate i.e. not to omit.
+Note in particular that this is the opposite of xarray conventions.
+If missing value handling needs to be configurable, it should follow scipy conventions.
+
+    nan_policy: {‘propagate’, ‘omit’, ‘raise’}
+    Defines how to handle input NaNs.
+    * propagate: if a NaN is present in the axis slice (e.g. row) along which the statistic is computed, the corresponding entry of the output will be NaN.
+    * omit: NaNs will be omitted when performing the calculation. If insufficient data remains in the axis slice along which the statistic is computed, the corresponding entry of the output will be NaN.
+    * raise: if a NaN is present, a ValueError will be raised.
+
+In complex multidimensional-data cases, it may be unclear to what axes propagation/omission etc. should apply to. Such cases warrant wider discussion.
diff --git a/docs/development/principles/naming.rst b/docs/development/principles/naming.rst
@@ -0,0 +1,29 @@
+Naming Conventions
+==================
+
+Consistent naming is important for making earthkit APIs predictable and
+easy to discover. Names should follow these principles:
+
+* Use British English spelling.
+* Prefer descriptive names that clearly communicate the purpose of a function,
+  class, or module. Variables or function arguments can be shorted.
+* Follow existing earthkit naming conventions rather than introducing new
+  patterns. If a convention is problematic, suggest changing it.
+* Use terminology that is consistent with the wider scientific Python
+  ecosystem where appropriate/possible e.g. earthkit-plots follows closely
+  matplotlib conventions.
+
+Function names should favour clarity over brevity. Avoid abbreviations unless they are
+well established and unambiguous.
+
+Function arguments on the other hand can be shorter.
+
+Before introducing a new name, check existing ``earthkit`` APIs and related
+packages for similar concepts. Similar operations should use the same names
+across modules.
+
+earthkit vs Earthkit
+--------------------
+
+In general, earthkit is lower caps when mentioning the repositories and software packages.
+It is capitalised when mentioning the project.