Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,19 @@ on:
tags: ["[0-9]*"]
pull_request:

# The repository default grants write; only the release job needs it.
permissions:
contents: read

# A new commit on a PR supersedes its running checks; main and tags always finish.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
test:
runs-on: ${{ matrix.os }}
timeout-minutes: 30 # the suite takes ~4 min; a hang must not hold a release
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -79,3 +89,23 @@ jobs:
name: dist
path: dist/
- uses: pypa/gh-action-pypi-publish@release/v1

release:
needs: publish
runs-on: ubuntu-latest
permissions:
contents: write # create the GitHub release
steps:
- uses: actions/checkout@v7
- name: Extract the release notes from the changelog
run: |
awk -v tag="$GITHUB_REF_NAME" '
index($0, "## [" tag "]") == 1 { found = 1; next }
found && /^## \[/ { exit }
found
' CHANGELOG.md > notes.md
test -s notes.md || { echo "no CHANGELOG.md section for $GITHUB_REF_NAME"; exit 1; }
- name: Create the GitHub release
env:
GH_TOKEN: ${{ github.token }}
run: gh release create "$GITHUB_REF_NAME" --verify-tag --title "$GITHUB_REF_NAME" --notes-file notes.md
37 changes: 29 additions & 8 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ Changes that break existing code are flagged **Breaking**.

## [Unreleased]

## [0.2.10] - 2026-10-16

### Added
- **`xdas.stalta`**, the short-term over long-term average characteristic function, feeding the existing `Trigger` so that detection no longer needs a model. Both windows are given in the units of the dimension. `method="recursive"` (default) and `"classic"` reproduce ObsPy's `recursive_sta_lta` and `classic_sta_lta` sample for sample, `"delayed"` ends the long window where the short one starts, and `"centered"` centres both windows for offline use. The atom is stateful and streams at a cost independent of the window lengths: chunked processing returns what a single call on the whole record returns, `flush()` releasing the tail `"centered"` holds back (@amaggi, @atrabattoni).
- **Aragon Photonics HDAS support.** The new `aragon` engine reads HDAS 3.0 `HDAS_StrainRate` HDF5 files, returning samples as stored. The time axis keeps the device stamps (to their float64 resolution) and declares the header sampling interval with the smallest tolerance they allow; `ctype={"time": "sampled"}` steps at the header interval and `"dense"` keeps every stamp. The distance axis comes from the header. HDF5 and tile virtualization (@atrabattoni).
Expand All @@ -19,9 +21,10 @@ Changes that break existing code are flagged **Breaking**.
- **A data collection mixing schemas prints as a tree.** Only a collection whose levels are all named, and named alike across branches, keeps its flat table; otherwise each such branch collapses into its dimensions and size, with its levels and leaf count hooked below. Leaves are now counted as arrays and tables (@atrabattoni).
- `simplify` on a coordinate with a declared sampling interval spends the budget it is given: a tolerance wider than the declared one thins past it and is declared on the result, so jittery clocks fold down to their real gaps; `tolerance=0` stays lossless (@atrabattoni).
- The `xinterp` requirement is raised to 0.2.1 (@atrabattoni).
- **The `dev` branch is retired**: development happens on `main`, so the latest version installs with `pip install "git+https://github.com/xdas-dev/xdas.git@main"`; installing from `@dev` no longer works. Releases are now published to PyPI and GitHub by CI from a version tag, and the PyPI page shows the README and project links (@atrabattoni).

### Fixed
- Fix opening files by wildcard on Windows, broken by a path-separator mismatch (#87) (@ClaudioStrumia).
- Fix `open_mfdatatree` on Windows, its path pattern not matching the backslash-separated paths found on disk (#87) (@ClaudioStrumia).
- Fix an unbounded `"memory_limit"` default where the physical memory cannot be read, as on Windows: a cgroup's "no limit" sentinel was taken for a real limit (#88) (@atrabattoni).
- Fix `StreamWriter` failing on Windows, its temporary filenames holding colons (#88) (@atrabattoni).
- Fix `simplify` dropping the declared sampling interval of an interpolated coordinate: a fusion that would break the rate is now refused (@atrabattoni).
Expand All @@ -30,18 +33,27 @@ Changes that break existing code are flagged **Breaking**.
## [0.2.9] - 2026-09-02

### Added
- **Tile-backed virtual arrays.** The new `xdas.virtual.tiles` module exposes a file archive as one lazy `TileArray`: slicing, concatenation and the numpy routines stay lazy, reductions stream, and a read touches only the tiles it overlaps. Select it with `vtype="tiles"` on any HDF5 engine — the default for Febus, always used by Silixa and ObsPy — or implement `Engine.load_tile`. Tile-backed arrays round-trip through the native netCDF format as a compact `__tiles__` group, relocatable by editing its header's single root path (@atrabattoni).

#### Tiles Backend
- **Tile-backed virtual arrays.** The new `xdas.virtual.tiles` module exposes a file archive as one lazy `TileArray`: slicing, concatenation and the numpy routines stay lazy, reductions stream, and a read touches only the tiles it overlaps. Select it with `vtype="tiles"` on any HDF5 engine — the default for Febus, always used by Silixa and ObsPy — or implement `Engine.load_tile` (@atrabattoni).
- Tile-backed arrays round-trip through the native netCDF format as a compact `__tiles__` group, relocatable by editing its header's single root path (@atrabattoni).
- **`xdas.sortby`** sorts a virtual data array along a dimension by coordinate value without reading data (@atrabattoni).

#### Atoms
- **Every atom has a functional form, and every function an atom form.** `xdas.resample(da, 50.0)` applies straight away; `xdas.resample(..., 50.0)`, seeded with `...`, is the atom behind it, ready to compose with `>>` and to be streamed. Numpy expressions extend a pipeline under the same seed: `20 * np.log10(np.abs(atom))` (@atrabattoni).
- **A processing vocabulary in physical units.** The top-level functions take the quantities of the measurement — a rate in Hz, corner frequencies, a kernel in seconds — so a parameter survives a change of sampling rate: `xdas.filter`, `resample`, `integrate`, `differentiate`, `stft`, `detrend`, `taper`, `hilbert`, `medfilt`, `sliding_mean_removal`, `rechunk`, `annotate`, `trigger` and `pick`. The SciPy-shaped ones stay in `xdas.signal` (@atrabattoni).
- **One `xdas.resample` for every way of changing a sampling rate.** The target is a `rate`, an `interval` or an `up`/`down` ratio, and `method="fir"` (default), `"iir"` or `"fft"` chooses how — `"fft"` alone reaches an arbitrary interval. The solver picks the simplest ratio within `tolerance`, not the closest; `snap=True` stays on the original grid (@atrabattoni).
- **Gap-aware chunked processing.** State carries across continuous chunks and is flushed at gaps and rate changes, eager calls splitting gappy input the same way, so a result no longer depends on the chunking. `flush()` drains buffered tails, and `xdas.testing.assert_chunk_invariant` asserts the property on a pipeline of your own (@atrabattoni).
- **`process()` on every atom, with source and sink auto-dispatch.** `pipeline.process(source, out=...)` infers both ends — an array, a path, a `DataCollection`, `xdas.watch(dir)` or a ZeroMQ address in; a directory, a `.csv`, an address or a writer out. An accumulation beyond the `"memory_limit"` entry raises rather than filling the machine (@atrabattoni).
- **Picking, end to end.** `Annotate` (replacing `MLPicker`) drives a SeisBench model with everything its weight set declares (overlap, stacking, blinding, preprocessing), overlapping GPU compute with transfers; `Trigger` gains per-phase thresholds; `xdas.pick(dc, model)` turns a tree of waveforms into one flat pick table (@atrabattoni).

#### Seismological Data
- **The `obspy` engine**, named for the library, not a format: decoding is `obspy.read`, so miniSEED, SAC, GSE2, SEG-2 and the rest go through it. Each contiguous `Trace` becomes one lazy `DataArray`, nested `network / station / location / channel`; files the miniseed engine rejected are now readable (@atrabattoni).
- **`xdas.stack`** collapses a level of a collection into an array dimension — `xd.stack(dc, "channel")` — lazily, with `tolerance` snapping near-identical sampling grids and `join="inner"/"outer"` for leaves that disagree (@atrabattoni).
- **`xdas.trim_overlaps`** drops the duplicated samples of an array or collection, keeping the later copy, or the earlier with `keep="first"`, at the manifest level so lazy arrays stay lazy (@atrabattoni).
- **`DataCollection.select`**, with `obspy.Stream.select` semantics: `dc.select(station="SX00*", channel="HH?")` (@atrabattoni).

#### General
- **Process pools for chunk ingress and egress.** `DataArrayLoader` and `DataArrayWriter` accept `pool="processes"` — on compressed archives, an order of magnitude faster than threads. Chunks travel through shared memory and arrive read-only; `pool="threads"` remains the default (@atrabattoni).
- `ZMQSubscriber.wait_until_subscribed()` returns once the publisher has greeted the subscription, so nothing published from then on is missed; `ZMQPublisher.wait_for_subscribers()` holds a replay back until its listeners have joined, and a `timeout` makes both subscribers raise on a stream gone quiet (@atrabattoni).
- `xdas.concat` can open a *new* dimension, checking the inputs agree elsewhere and promoting the scalar coordinates that vary: `xd.concat(traces, "channel")` (@atrabattoni).
Expand All @@ -50,23 +62,31 @@ Changes that break existing code are flagged **Breaking**.
- `xdas.config` gains `"scan_workers"` beside `"n_workers"` (@atrabattoni).

### Changed
- **The machine-parameter atoms become an expert layer.** `LFilter`, `SOSFilter`, `DownSample`, `UpSample` and the new fused `Polyphase` move to `xdas.atoms.kernel`, still importable from `xdas.atoms`: the processing vocabulary designs them from the data. Resampling rides `Polyphase`, 2.6–8.7× faster than the chain it replaces (@atrabattoni).

#### Tiles Backend
- Scanning no longer grows in memory with the file count: a 23-million-file archive opens in ~1 GB (@atrabattoni).
- **A data collection prints as a table of its leaves**: one row per leaf with the keys that address it, its shape and its size in memory, plus a summary line (@atrabattoni).

#### Atoms
- **The machine-parameter atoms become an expert layer.** `LFilter`, `SOSFilter`, `DownSample`, `UpSample` and the new fused `Polyphase` move to `xdas.atoms.kernel`, still importable from `xdas.atoms`: the processing vocabulary designs them from the data. Resampling rides `Polyphase`, 2.6–8.7× faster than the chain it replaces (@atrabattoni).
- **An atom prints as one line, and a pipeline as one line per stage**, showing the parameters its constructor was given with the defaults left out — `Filter(freq=(None, 10.0), ftype='fir')`. A model or a set of coefficients is summarised as `<PhaseNet>` or `<ndarray>` (@atrabattoni).
- **Breaking:** `Trigger` moved to `xdas.atoms.detect`, with a lowercase twin `xdas.trigger`; the module remains importable and `find_picks` is unchanged. Its `dim` default is now `"time"` instead of `"last"` (@atrabattoni).
- **Breaking:** the `xdas.atoms.signal` module is split into `xdas.atoms.kernel` and `xdas.atoms.operations`; its atoms are still importable from `xdas.atoms` (@atrabattoni).

#### Seismological Data
- **Breaking: opening a seismological file without naming an engine returns a nested collection, not a stacked array.** `xd.concat(traces, "channel")` is the one-liner back, and `engine="miniseed"` still gives the old shape (@atrabattoni).

#### General
- **A data collection prints as a table of its leaves**: one row per leaf with the keys that address it, its shape and its size in memory, plus a summary line (@atrabattoni).
- Format-specific engine parameters are constructor arguments, validated up front: a misspelled keyword now raises (@atrabattoni).
- **Parallelism sized to the work, not to the machine.** Both were `os.cpu_count()`, which made small work slower. Threads are now sized against the array they are given, and a small scan stays in the calling process (@atrabattoni).
- A ZeroMQ publisher answers each new subscription with a greeting, so subscribing no longer races the stream (@atrabattoni).
- Acquisitions interleaved in time now group by compatibility, one array each, instead of splitting at every alternation (@atrabattoni).
- **`simplify` is linear in the number of gaps** (@atrabattoni).
- When no engine can open a file, the error lists what each engine that recognised it said (@atrabattoni).
- **Breaking:** the numpy requirement is raised to 2.3 (@atrabattoni).
- **Breaking: opening a seismological file without naming an engine returns a nested collection, not a stacked array.** `xd.concat(traces, "channel")` is the one-liner back, and `engine="miniseed"` still gives the old shape (@atrabattoni).
- **Breaking: the innermost level of a collection is named `record`**, whatever it holds: `dc.query(trace=0)` becomes `dc.query(record=0)`. A collection saved before this reads back with its old level names (@atrabattoni).
- **Breaking:** `DataCollection.query` raises a `KeyError` on an indexer naming no level of the collection, instead of silently returning everything unchanged (@atrabattoni).
- **Breaking:** custom engines must subclass `xdas.io.Engine`: passing a bare read function as `engine` now raises a `TypeError` (@atrabattoni).
- **Breaking:** `Trigger` moved to `xdas.atoms.detect`, with a lowercase twin `xdas.trigger`; the module remains importable and `find_picks` is unchanged. Its `dim` default is now `"time"` instead of `"last"` (@atrabattoni).
- **Breaking:** the `xdas.atoms.signal` module is split into `xdas.atoms.kernel` and `xdas.atoms.operations`; its atoms are still importable from `xdas.atoms` (@atrabattoni).

### Deprecated
- `MLPicker` and `xdas.mlpicker` are deprecated in favour of `Annotate` and `Picker`; they remain as aliases until 0.4 (@atrabattoni).
Expand Down Expand Up @@ -248,7 +268,8 @@ Changes that break existing code are flagged **Breaking**.

Initial stable version.

[unreleased]: https://github.com/xdas-dev/xdas/compare/0.2.9...HEAD
[unreleased]: https://github.com/xdas-dev/xdas/compare/0.2.10...HEAD
[0.2.10]: https://github.com/xdas-dev/xdas/compare/0.2.9...0.2.10
[0.2.9]: https://github.com/xdas-dev/xdas/compare/0.2.8...0.2.9
[0.2.8]: https://github.com/xdas-dev/xdas/compare/0.2.7...0.2.8
[0.2.7]: https://github.com/xdas-dev/xdas/compare/0.2.6...0.2.7
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
<div align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="./docs/_static/logo-dark.png">
<source media="(prefers-color-scheme: light)" srcset="./docs/_static/logo-light.png">
<img alt="Xdas Logo" height="250px">
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/xdas-dev/xdas/main/docs/_static/logo-dark.png">
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/xdas-dev/xdas/main/docs/_static/logo-light.png">
<img alt="Xdas Logo" height="250px" src="https://raw.githubusercontent.com/xdas-dev/xdas/main/docs/_static/logo-light.png">
</picture>
</div>

Expand Down
4 changes: 2 additions & 2 deletions docs/contribute.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ To do this you can either use git in the [terminal](https://git-scm.com/docs/git

## How to write a changelog entry

`CHANGELOG.md` follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Add one line per user-visible change under `[Unreleased]`, in the section that fits: `Added`, `Changed`, `Deprecated`, `Removed`, `Fixed` or `Security`. Write it for users, not reviewers: say what changed for them, start with **Breaking:** when existing code stops working, and credit yourself at the end, e.g. `(@your-github-handle)`. Purely internal changes (tests, CI, refactoring) need no entry.
`CHANGELOG.md` follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Add one line per user-visible change under `[Unreleased]`, in the section that fits: `Added`, `Changed`, `Deprecated`, `Removed`, `Fixed` or `Security`. A large release may group a section's entries under `####` topic headings (e.g. `#### Atoms`). Write it for users, not reviewers: say what changed for them, start with **Breaking:** when existing code stops working, and credit yourself at the end, e.g. `(@your-github-handle)`. Purely internal changes (tests, CI, refactoring) need no entry.


## How to write a docstring with examples
Expand Down Expand Up @@ -175,4 +175,4 @@ Releases are cut from `main` by a maintainer:
git tag X.Y.Z
git push origin X.Y.Z
```
The CI then runs the tests again, checks that the tag matches `__version__`, builds the package and publishes it to PyPI.
The CI then runs the tests again, checks that the tag matches `__version__`, builds the package, publishes it to PyPI once a maintainer approves the `pypi` deployment, and finally creates the GitHub release with the version's `CHANGELOG.md` section as notes (which Zenodo archives).
7 changes: 7 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "xdas"
dynamic = ["version"]
readme = "README.md"
requires-python = ">= 3.11"
authors = [
{ name = "Alister Trabattoni", email = "alister.trabattoni@gmail.com" },
Expand Down Expand Up @@ -43,6 +44,12 @@ docs = [
tests = ["dascore", "dask<2025.4.0", "psutil", "seisbench", "torch>=2.7"]

# Single source of truth for the version: xdas/__init__.py
[project.urls]
Homepage = "https://github.com/xdas-dev/xdas"
Documentation = "https://xdas.readthedocs.io"
Changelog = "https://github.com/xdas-dev/xdas/blob/main/CHANGELOG.md"
Issues = "https://github.com/xdas-dev/xdas/issues"

[tool.setuptools.dynamic]
version = { attr = "xdas.__version__" }

Expand Down
2 changes: 1 addition & 1 deletion xdas/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
for common DAS instrument formats.
"""

__version__ = "0.2.9"
__version__ = "0.2.10"

__all__ = [ # noqa: RUF022 - grouped by kind, not alphabetically
# submodules
Expand Down
17 changes: 0 additions & 17 deletions xdas/processing/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -1732,16 +1732,6 @@ class ZMQPublisher(SubscriptionTracker):
nsubscribers : int
The number of currently subscribed peers.

Methods
-------
submit(da)
Send a DataArray over ZeroMQ.
wait_for_subscribers(count, timeout)
Blocks until *count* peers are subscribed, so that nothing published
afterwards is dropped.
close()
Release the socket and its context.

Examples
--------
>>> import xdas as xd
Expand Down Expand Up @@ -1826,13 +1816,6 @@ class ZMQSubscriber(ZMQEndpoint):
How many seconds to wait at most for each packet. None, the default,
waits forever.

Methods
-------
wait_until_subscribed()
Block until the publisher has registered this subscription.
close()
Release the socket and its context.

Examples
--------
>>> import threading
Expand Down
Loading