Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/code-coverage.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
name: Code Coverage

on: [push]
on:
push:
branches: [main, dev]
pull_request:

jobs:
test:
Expand Down
5 changes: 4 additions & 1 deletion .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
name: tests

on: [push]
on:
push:
branches: [main, dev]
pull_request:

jobs:
build:
Expand Down
11 changes: 7 additions & 4 deletions docs/api/atoms.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ data it applies eagerly, called on `...` it returns the atom.
rechunk
resample
sliding_mean_removal
stalta
stft
taper
trigger
Expand All @@ -144,16 +145,18 @@ data it applies eagerly, called on `...` it returns the atom.

## Detection and picking

`Annotate` runs a SeisBench model window by window, `Trigger` turns the
characteristic function it produces into a pick table, and `Picker` is the
whole pipeline a weight set describes — its own filter, its own sampling rate,
`STALTA` and `Annotate` build a characteristic function — the first from the
amplitude of the signal itself, the second by running a SeisBench model window
by window — `Trigger` turns either one into a pick table, and `Picker` is the
whole pipeline a weight set describes: its own filter, its own sampling rate,
its own per-phase thresholds. Each has a lowercase functional twin at the top
level of `xdas` (`xdas.annotate`, `xdas.trigger`, `xdas.pick`).
level of `xdas` (`xdas.stalta`, `xdas.annotate`, `xdas.trigger`, `xdas.pick`).

```{eval-rst}
.. autosummary::
:toctree: ../_autosummary

STALTA
Annotate
Trigger
Picker
Expand Down
1 change: 1 addition & 0 deletions docs/release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
## 0.2.10 (unreleased)

### New Features
- **`xdas.stalta`**, the short-term over long-term average characteristic function, feeding the existing `Trigger` so that detection no longer needs a model. Both windows are given in the units of the dimension. `method="recursive"` (default) and `"classic"` reproduce ObsPy's `recursive_sta_lta` and `classic_sta_lta` sample for sample, `"delayed"` ends the long window where the short one starts, and `"centered"` centres both windows for offline use. The atom is stateful and streams at a cost independent of the window lengths: chunked processing returns what a single call on the whole record returns, `flush()` releasing the tail `"centered"` holds back (@amaggi, @atrabattoni).
- **Aragon Photonics HDAS support.** The new `aragon` engine reads HDAS 3.0 `HDAS_StrainRate` HDF5 files, returning samples as stored. The time axis keeps the device stamps (to their float64 resolution) and declares the header sampling interval with the smallest tolerance they allow; `ctype={"time": "sampled"}` steps at the header interval and `"dense"` keeps every stamp. The distance axis comes from the header. HDF5 and tile virtualization (@atrabattoni).
- Add a `"fosina"` alias of the `"prodml"` engine, and list in the user guide the instruments known to write ProdML (@atrabattoni).

Expand Down
3 changes: 2 additions & 1 deletion docs/user-guide/pipeline/atoms.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,8 @@ energy
| {py:func}`~xdas.hilbert` | analytic signal |
| {py:func}`~xdas.medfilt`, {py:func}`~xdas.sliding_mean_removal` | kernels in seconds or meters |
| {py:func}`~xdas.rechunk` | a streaming-cadence knob, not an operation |
| {py:func}`~xdas.annotate`, {py:func}`~xdas.trigger`, {py:func}`~xdas.pick` | machine-learning picking (see [](picking.md)) |
| {py:func}`~xdas.stalta` | short-term over long-term average (ObsPy's recursive, classic, delayed, or centered), windows in seconds |
| {py:func}`~xdas.annotate`, {py:func}`~xdas.trigger`, {py:func}`~xdas.pick` | detection and picking (see [](picking.md)) |

Each has an atom behind it — {py:class}`~xdas.atoms.Filter`,
{py:class}`~xdas.atoms.Resample`, and so on — reached by seeding with `...`.
Expand Down
63 changes: 63 additions & 0 deletions docs/user-guide/pipeline/picking.md
Original file line number Diff line number Diff line change
Expand Up @@ -288,3 +288,66 @@ across that band. When the data already arrives at the model's training rate
the stage does nothing and the two agree exactly; when a resample is needed the
picks can diverge. Pass `resample=False` to drop the stage, or to compare the
two implementations on identical waveforms.

## Detecting without a model

Not every detection needs a network of weights. {py:func}`~xdas.stalta` is the
classic amplitude detector — a short average of the squared signal over a long
one — and it feeds the very same {py:class}`~xdas.atoms.Trigger`, so the pick
table and everything downstream of it are unchanged. Both window lengths are
given in the units of the dimension, seconds here, so the same atom applies
whatever the sampling rate:

```{code-cell}
import numpy as np

rng = np.random.default_rng(42)
data = rng.normal(size=(6000, 4))
data[2000:2100] += 8.0 * np.exp(-np.arange(100) / 25.0)[:, None]
noisy = xd.DataArray(
data,
{
"time": {
"tie_indices": [0, 5999],
"tie_values": [0.0, 59.99],
"sampling_interval": 0.01,
},
"distance": np.arange(4) * 10.0,
},
)

xd.trigger(xd.stalta(noisy, sta=0.5, lta=10.0), thresh=4.0)
```

The default `method="recursive"` is the Evans and Allen exponential average,
and `"classic"` the trailing boxcar one: both reproduce their
[ObsPy](https://docs.obspy.org/packages/autogen/obspy.signal.trigger.html)
counterparts sample for sample, including the zeros reported while the long
window is not yet full. `"delayed"` ends the long window where the short one
starts, so that a transient does not raise its own reference.

All three are causal: they only ever see the past, and they are the methods a
real-time detector can use. Their peak trails the transient — by about half
the short window for the boxcar ones, which only peak once that window is full
of it. Offline, `method="centered"` averages symmetrically about the sample it
produces, which brings the peak back onto the transient at the price of
needing samples from the future:

```{code-cell}
xd.trigger(xd.stalta(noisy, sta=0.5, lta=10.0, method="centered"), thresh=4.0)
```

The choice moves pick times, so it is a property of the detector rather than a
tuning knob — the same threshold on the same record gives different times
under different methods.

Like every atom, it is stateful: an hour arriving in ten-second chunks gives
exactly what the hour gives in one piece, seam for seam, at a cost that does
not depend on the window lengths. With `"centered"` the samples whose window is
not yet complete are held back, and `flush()` releases them at the end of the
stream — which the streaming machinery of [](processing.md) calls for you:

```{code-cell}
pipeline = xd.stalta(..., sta=0.5, lta=10.0) >> xd.trigger(..., thresh=4.0)
pipeline.process(noisy, chunks={"time": 500})
```
218 changes: 217 additions & 1 deletion tests/test_atoms_detect.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import pytest

import xdas as xd
from xdas.atoms import Trigger
from xdas.atoms import STALTA, Trigger


def generate():
Expand Down Expand Up @@ -455,3 +455,219 @@ def test_each_lane_keeps_its_own_identity_and_time_base(self):
assert sorted(zip(chunked["distance"], chunked["time"])) == sorted(
zip(eager["distance"], eager["time"])
)


STALTA_METHODS = ["recursive", "classic", "delayed", "centered"]
BOXCAR_METHODS = ["classic", "delayed", "centered"]


def generate_stalta(nt=6000, nx=3, seed=42, delta=0.01):
"""A noisy record with three transients, for the characteristic function."""
rng = np.random.default_rng(seed)
data = rng.normal(0.0, 1.0, (nt, nx))
for onset in (1500, 3200, 4700):
if onset + 100 <= nt:
data[onset : onset + 100] += 15.0 * np.exp(-np.arange(100) / 25.0)[:, None]
return xd.DataArray(
data=data,
coords={
"time": {
"tie_indices": [0, nt - 1],
"tie_values": [0.0, (nt - 1) * delta],
"sampling_interval": delta,
},
"distance": np.arange(nx) * 10.0,
},
)


def boxcar_reference(x, nsta, nlta, method):
"""Direct sum of every window, zero where one is incomplete."""
energy = np.square(x)
n = len(x)
out = np.zeros(n)
for i in range(n):
match method:
case "classic":
ends = (i, i)
case "delayed":
ends = (i, i - nsta)
case "centered":
ends = (i + nsta // 2, i + nlta // 2)
windows = [(ends[0] - nsta + 1, ends[0]), (ends[1] - nlta + 1, ends[1])]
if all(lo >= 0 and hi < n for lo, hi in windows):
(a, b), (c, d) = windows
out[i] = energy[a : b + 1].mean() / energy[c : d + 1].mean()
return out


class TestSTALTA:
@pytest.mark.parametrize("sta, lta", [(0.5, 10.0), (0.21, 3.0)])
def test_recursive_matches_obspy(self, sta, lta):
from obspy.signal.trigger import recursive_sta_lta

da = generate_stalta()
result = STALTA(sta, lta, method="recursive")(da)
nsta, nlta = round(sta / 0.01), round(lta / 0.01)
for lane in range(da.sizes["distance"]):
expected = recursive_sta_lta(da.values[:, lane], nsta, nlta)
# the same recursion, but a compiler may fuse its multiply-add
# (arm64 does), so only round-off apart
np.testing.assert_allclose(result.values[:, lane], expected, rtol=1e-12)
np.testing.assert_array_equal(result.values[:, lane] == 0, expected == 0)

@pytest.mark.parametrize("sta, lta", [(0.5, 10.0), (0.21, 3.0)])
def test_classic_matches_obspy(self, sta, lta):
from obspy.signal.trigger import classic_sta_lta

da = generate_stalta()
result = STALTA(sta, lta, method="classic")(da)
nsta, nlta = round(sta / 0.01), round(lta / 0.01)
for lane in range(da.sizes["distance"]):
expected = classic_sta_lta(da.values[:, lane], nsta, nlta)
np.testing.assert_allclose(result.values[:, lane], expected, rtol=1e-10)
# the same zero warm-up, sample for sample
np.testing.assert_array_equal(result.values[:, lane] == 0, expected == 0)

@pytest.mark.parametrize("method", BOXCAR_METHODS)
@pytest.mark.parametrize("sta, lta", [(0.5, 10.0), (0.2, 3.0)])
def test_boxcar_matches_direct_sums(self, method, sta, lta):
da = generate_stalta()
result = STALTA(sta, lta, method=method)(da)
nsta, nlta = round(sta / 0.01), round(lta / 0.01)
expected = boxcar_reference(da.values[:, 0], nsta, nlta, method)
np.testing.assert_allclose(result.values[:, 0], expected, rtol=1e-10)

@pytest.mark.parametrize("method", STALTA_METHODS)
def test_dim_on_any_axis(self, method):
da = generate_stalta()
expected = STALTA(0.5, 10.0, method=method)(da)
result = STALTA(0.5, 10.0, method=method)(da.transpose("distance", "time"))
assert result.dims == ("distance", "time")
np.testing.assert_array_equal(result.values, expected.values.T)

@pytest.mark.parametrize("method", STALTA_METHODS)
def test_chunk_invariant(self, method):
da = generate_stalta()
atom = STALTA(0.5, 10.0, method=method)
xd.testing.assert_chunk_invariant(atom, da, chunks={"time": 1000}, gaps=2)
# chunks far shorter than the long window, and than what "centered"
# holds back
xd.testing.assert_chunk_invariant(atom, da, chunks={"time": 37}, cuts=0)

@pytest.mark.parametrize("method", STALTA_METHODS)
def test_chunked_along_another_dim(self, method):
da = generate_stalta()
expected = STALTA(0.5, 10.0, method=method)(da)
result = STALTA(0.5, 10.0, method=method).process(da, chunks={"distance": 1})
np.testing.assert_array_equal(result.values, expected.values)

@pytest.mark.parametrize("method", STALTA_METHODS)
def test_scale_invariant(self, method):
# DAS strain rates are ~1e-9: their energy must not hit the floor
da = generate_stalta()
expected = STALTA(0.5, 10.0, method=method)(da).values
result = STALTA(0.5, 10.0, method=method)(da * 1e-9).values
np.testing.assert_allclose(result, expected, rtol=1e-9)

@pytest.mark.parametrize("method", BOXCAR_METHODS)
def test_round_off_does_not_outlive_a_loud_transient(self, method):
# A burst 1e9 times louder than what follows: a cumulative sum keeps
# its round-off for the rest of the trace, the running sums for at
# most two windows.
da = generate_stalta(nt=12000, nx=1)
data = da.values.copy()
data[2000:2100] *= 1e6
data[3000:] *= 1e-3
da = da.copy(data=data)
nsta, nlta = 50, 1000
result = STALTA(0.5, 10.0, method=method)(da).values[:, 0]
expected = boxcar_reference(data[:, 0], nsta, nlta, method)
clean = slice(2100 + 2 * (nsta + nlta), None)
np.testing.assert_allclose(result[clean], expected[clean], rtol=1e-8)

@pytest.mark.parametrize("method", ["recursive", "classic", "delayed"])
def test_causal_methods_ignore_the_future(self, method):
da = generate_stalta()
expected = STALTA(0.5, 10.0, method=method)(da).values
data = da.values.copy()
data[2500:] = 100.0
result = STALTA(0.5, 10.0, method=method)(da.copy(data=data)).values
np.testing.assert_array_equal(result[:2500], expected[:2500])
assert not np.array_equal(result[2500:], expected[2500:])

@pytest.mark.parametrize("method", ["recursive", "classic", "delayed"])
def test_causal_methods_hold_nothing_back(self, method):
da = generate_stalta()
atom = STALTA(0.5, 10.0, method=method)
for chunk in xd.split(da, 7, dim="time"):
out = atom(chunk, chunk_dim="time")
assert out.sizes["time"] == chunk.sizes["time"]
assert atom.flush() == []

def test_centered_holds_back_half_the_long_window(self):
da = generate_stalta()
atom = STALTA(0.5, 10.0, method="centered")
outs = [atom(chunk, chunk_dim="time") for chunk in xd.split(da, 4, dim="time")]
emitted = xd.concat(outs, "time").sizes["time"]
assert emitted == da.sizes["time"] - 1000 // 2
(tail,) = atom.flush()
assert tail.sizes["time"] == 1000 // 2
np.testing.assert_array_equal(tail.values, 0.0)
assert atom.flush() == []

def test_centered_onset_is_earlier_than_causal(self):
da = generate_stalta()
causal = STALTA(0.5, 10.0, method="classic")(da)
centered = STALTA(0.5, 10.0, method="centered")(da)
assert int(causal.values[:, 0].argmax()) > int(centered.values[:, 0].argmax())

@pytest.mark.parametrize("method", STALTA_METHODS)
def test_record_shorter_than_the_windows(self, method):
# nothing is ever complete: all zeros, chunked or not
da = generate_stalta(nt=300)
expected = STALTA(0.5, 10.0, method=method)(da)
np.testing.assert_array_equal(expected.values, 0.0)
result = STALTA(0.5, 10.0, method=method).process(da, chunks={"time": 40})
np.testing.assert_array_equal(result.values, expected.values)

def test_flush_without_data_is_empty(self):
# uninitialised, and initialised by an eager (unchunked) call: neither
# holds anything back
atom = STALTA(0.5, 10.0, method="centered")
assert atom.flush() == []
atom(generate_stalta())
assert atom.flush() == []

@pytest.mark.parametrize("method", STALTA_METHODS)
def test_parallel_matches_serial(self, method):
# each worker updates its own slice of lanes of the carried sums
da = generate_stalta(nx=7)
expected = STALTA(0.5, 10.0, method=method, parallel=False)(da)
result = STALTA(0.5, 10.0, method=method, parallel=3)(da)
np.testing.assert_array_equal(result.values, expected.values)
atom = STALTA(0.5, 10.0, method=method, parallel=3)
chunked = atom.process(da, chunks={"time": 2500})
np.testing.assert_array_equal(chunked.values, expected.values)

def test_twin_and_pipeline(self):
da = generate_stalta()
np.testing.assert_array_equal(
xd.stalta(da, sta=0.5, lta=10.0).values, STALTA(0.5, 10.0)(da).values
)
assert isinstance(xd.stalta(..., sta=0.5, lta=10.0), STALTA)
picks = da >> (xd.stalta(..., sta=0.5, lta=10.0) >> xd.trigger(..., thresh=5.0))
assert len(picks) > 0

def test_invalid_parameters(self):
with pytest.raises(ValueError, match="`method` must be one of"):
STALTA(0.5, 10.0, method="causal")
with pytest.raises(ValueError, match="strictly positive"):
STALTA(0.0, 1.0)
with pytest.raises(ValueError, match="longer than"):
STALTA(1.0, 0.5)
da = generate_stalta()
with pytest.raises(ValueError, match="shorter than one sample"):
STALTA(0.004, 10.0)(da)
with pytest.raises(ValueError, match="more samples than"):
STALTA(0.5, 0.504)(da)
3 changes: 2 additions & 1 deletion xdas/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@
"rechunk",
"resample",
"sliding_mean_removal",
"stalta",
"stft",
"taper",
"trigger",
Expand All @@ -97,7 +98,7 @@
# `sys.modules` and does not rebind the attribute: the lowercase twin below
# stays `xdas.trigger` for everyone.
from . import trigger as _trigger_module # noqa: F401 isort: skip
from .atoms.detect import trigger
from .atoms.detect import stalta, trigger
from .atoms.kernel import rechunk
from .atoms.ml import annotate, mlpicker, pick
from .atoms.operations import (
Expand Down
Loading
Loading