diff --git a/.github/workflows/code-coverage.yaml b/.github/workflows/code-coverage.yaml index 6f7ca3bf..14151ed2 100644 --- a/.github/workflows/code-coverage.yaml +++ b/.github/workflows/code-coverage.yaml @@ -1,6 +1,9 @@ name: Code Coverage -on: [push] +on: + push: + branches: [main, dev] + pull_request: jobs: test: diff --git a/.github/workflows/tests.yaml b/.github/workflows/tests.yaml index eca09ad2..c4400f67 100644 --- a/.github/workflows/tests.yaml +++ b/.github/workflows/tests.yaml @@ -1,6 +1,9 @@ name: tests -on: [push] +on: + push: + branches: [main, dev] + pull_request: jobs: build: diff --git a/docs/api/atoms.md b/docs/api/atoms.md index 348fa115..fb273b03 100644 --- a/docs/api/atoms.md +++ b/docs/api/atoms.md @@ -133,6 +133,7 @@ data it applies eagerly, called on `...` it returns the atom. rechunk resample sliding_mean_removal + stalta stft taper trigger @@ -144,16 +145,18 @@ data it applies eagerly, called on `...` it returns the atom. ## Detection and picking -`Annotate` runs a SeisBench model window by window, `Trigger` turns the -characteristic function it produces into a pick table, and `Picker` is the -whole pipeline a weight set describes — its own filter, its own sampling rate, +`STALTA` and `Annotate` build a characteristic function — the first from the +amplitude of the signal itself, the second by running a SeisBench model window +by window — `Trigger` turns either one into a pick table, and `Picker` is the +whole pipeline a weight set describes: its own filter, its own sampling rate, its own per-phase thresholds. Each has a lowercase functional twin at the top -level of `xdas` (`xdas.annotate`, `xdas.trigger`, `xdas.pick`). +level of `xdas` (`xdas.stalta`, `xdas.annotate`, `xdas.trigger`, `xdas.pick`). ```{eval-rst} .. autosummary:: :toctree: ../_autosummary + STALTA Annotate Trigger Picker diff --git a/docs/release-notes.md b/docs/release-notes.md index 90fb7c74..c38ba6dd 100644 --- a/docs/release-notes.md +++ b/docs/release-notes.md @@ -3,6 +3,7 @@ ## 0.2.10 (unreleased) ### New Features +- **`xdas.stalta`**, the short-term over long-term average characteristic function, feeding the existing `Trigger` so that detection no longer needs a model. Both windows are given in the units of the dimension. `method="recursive"` (default) and `"classic"` reproduce ObsPy's `recursive_sta_lta` and `classic_sta_lta` sample for sample, `"delayed"` ends the long window where the short one starts, and `"centered"` centres both windows for offline use. The atom is stateful and streams at a cost independent of the window lengths: chunked processing returns what a single call on the whole record returns, `flush()` releasing the tail `"centered"` holds back (@amaggi, @atrabattoni). - **Aragon Photonics HDAS support.** The new `aragon` engine reads HDAS 3.0 `HDAS_StrainRate` HDF5 files, returning samples as stored. The time axis keeps the device stamps (to their float64 resolution) and declares the header sampling interval with the smallest tolerance they allow; `ctype={"time": "sampled"}` steps at the header interval and `"dense"` keeps every stamp. The distance axis comes from the header. HDF5 and tile virtualization (@atrabattoni). - Add a `"fosina"` alias of the `"prodml"` engine, and list in the user guide the instruments known to write ProdML (@atrabattoni). diff --git a/docs/user-guide/pipeline/atoms.md b/docs/user-guide/pipeline/atoms.md index 34c94a0f..539ab146 100644 --- a/docs/user-guide/pipeline/atoms.md +++ b/docs/user-guide/pipeline/atoms.md @@ -86,7 +86,8 @@ energy | {py:func}`~xdas.hilbert` | analytic signal | | {py:func}`~xdas.medfilt`, {py:func}`~xdas.sliding_mean_removal` | kernels in seconds or meters | | {py:func}`~xdas.rechunk` | a streaming-cadence knob, not an operation | -| {py:func}`~xdas.annotate`, {py:func}`~xdas.trigger`, {py:func}`~xdas.pick` | machine-learning picking (see [](picking.md)) | +| {py:func}`~xdas.stalta` | short-term over long-term average (ObsPy's recursive, classic, delayed, or centered), windows in seconds | +| {py:func}`~xdas.annotate`, {py:func}`~xdas.trigger`, {py:func}`~xdas.pick` | detection and picking (see [](picking.md)) | Each has an atom behind it — {py:class}`~xdas.atoms.Filter`, {py:class}`~xdas.atoms.Resample`, and so on — reached by seeding with `...`. diff --git a/docs/user-guide/pipeline/picking.md b/docs/user-guide/pipeline/picking.md index 88cefa04..50e6314e 100644 --- a/docs/user-guide/pipeline/picking.md +++ b/docs/user-guide/pipeline/picking.md @@ -288,3 +288,66 @@ across that band. When the data already arrives at the model's training rate the stage does nothing and the two agree exactly; when a resample is needed the picks can diverge. Pass `resample=False` to drop the stage, or to compare the two implementations on identical waveforms. + +## Detecting without a model + +Not every detection needs a network of weights. {py:func}`~xdas.stalta` is the +classic amplitude detector — a short average of the squared signal over a long +one — and it feeds the very same {py:class}`~xdas.atoms.Trigger`, so the pick +table and everything downstream of it are unchanged. Both window lengths are +given in the units of the dimension, seconds here, so the same atom applies +whatever the sampling rate: + +```{code-cell} +import numpy as np + +rng = np.random.default_rng(42) +data = rng.normal(size=(6000, 4)) +data[2000:2100] += 8.0 * np.exp(-np.arange(100) / 25.0)[:, None] +noisy = xd.DataArray( + data, + { + "time": { + "tie_indices": [0, 5999], + "tie_values": [0.0, 59.99], + "sampling_interval": 0.01, + }, + "distance": np.arange(4) * 10.0, + }, +) + +xd.trigger(xd.stalta(noisy, sta=0.5, lta=10.0), thresh=4.0) +``` + +The default `method="recursive"` is the Evans and Allen exponential average, +and `"classic"` the trailing boxcar one: both reproduce their +[ObsPy](https://docs.obspy.org/packages/autogen/obspy.signal.trigger.html) +counterparts sample for sample, including the zeros reported while the long +window is not yet full. `"delayed"` ends the long window where the short one +starts, so that a transient does not raise its own reference. + +All three are causal: they only ever see the past, and they are the methods a +real-time detector can use. Their peak trails the transient — by about half +the short window for the boxcar ones, which only peak once that window is full +of it. Offline, `method="centered"` averages symmetrically about the sample it +produces, which brings the peak back onto the transient at the price of +needing samples from the future: + +```{code-cell} +xd.trigger(xd.stalta(noisy, sta=0.5, lta=10.0, method="centered"), thresh=4.0) +``` + +The choice moves pick times, so it is a property of the detector rather than a +tuning knob — the same threshold on the same record gives different times +under different methods. + +Like every atom, it is stateful: an hour arriving in ten-second chunks gives +exactly what the hour gives in one piece, seam for seam, at a cost that does +not depend on the window lengths. With `"centered"` the samples whose window is +not yet complete are held back, and `flush()` releases them at the end of the +stream — which the streaming machinery of [](processing.md) calls for you: + +```{code-cell} +pipeline = xd.stalta(..., sta=0.5, lta=10.0) >> xd.trigger(..., thresh=4.0) +pipeline.process(noisy, chunks={"time": 500}) +``` diff --git a/tests/test_atoms_detect.py b/tests/test_atoms_detect.py index 5ad2464a..f786b751 100644 --- a/tests/test_atoms_detect.py +++ b/tests/test_atoms_detect.py @@ -10,7 +10,7 @@ import pytest import xdas as xd -from xdas.atoms import Trigger +from xdas.atoms import STALTA, Trigger def generate(): @@ -455,3 +455,219 @@ def test_each_lane_keeps_its_own_identity_and_time_base(self): assert sorted(zip(chunked["distance"], chunked["time"])) == sorted( zip(eager["distance"], eager["time"]) ) + + +STALTA_METHODS = ["recursive", "classic", "delayed", "centered"] +BOXCAR_METHODS = ["classic", "delayed", "centered"] + + +def generate_stalta(nt=6000, nx=3, seed=42, delta=0.01): + """A noisy record with three transients, for the characteristic function.""" + rng = np.random.default_rng(seed) + data = rng.normal(0.0, 1.0, (nt, nx)) + for onset in (1500, 3200, 4700): + if onset + 100 <= nt: + data[onset : onset + 100] += 15.0 * np.exp(-np.arange(100) / 25.0)[:, None] + return xd.DataArray( + data=data, + coords={ + "time": { + "tie_indices": [0, nt - 1], + "tie_values": [0.0, (nt - 1) * delta], + "sampling_interval": delta, + }, + "distance": np.arange(nx) * 10.0, + }, + ) + + +def boxcar_reference(x, nsta, nlta, method): + """Direct sum of every window, zero where one is incomplete.""" + energy = np.square(x) + n = len(x) + out = np.zeros(n) + for i in range(n): + match method: + case "classic": + ends = (i, i) + case "delayed": + ends = (i, i - nsta) + case "centered": + ends = (i + nsta // 2, i + nlta // 2) + windows = [(ends[0] - nsta + 1, ends[0]), (ends[1] - nlta + 1, ends[1])] + if all(lo >= 0 and hi < n for lo, hi in windows): + (a, b), (c, d) = windows + out[i] = energy[a : b + 1].mean() / energy[c : d + 1].mean() + return out + + +class TestSTALTA: + @pytest.mark.parametrize("sta, lta", [(0.5, 10.0), (0.21, 3.0)]) + def test_recursive_matches_obspy(self, sta, lta): + from obspy.signal.trigger import recursive_sta_lta + + da = generate_stalta() + result = STALTA(sta, lta, method="recursive")(da) + nsta, nlta = round(sta / 0.01), round(lta / 0.01) + for lane in range(da.sizes["distance"]): + expected = recursive_sta_lta(da.values[:, lane], nsta, nlta) + # the same recursion, but a compiler may fuse its multiply-add + # (arm64 does), so only round-off apart + np.testing.assert_allclose(result.values[:, lane], expected, rtol=1e-12) + np.testing.assert_array_equal(result.values[:, lane] == 0, expected == 0) + + @pytest.mark.parametrize("sta, lta", [(0.5, 10.0), (0.21, 3.0)]) + def test_classic_matches_obspy(self, sta, lta): + from obspy.signal.trigger import classic_sta_lta + + da = generate_stalta() + result = STALTA(sta, lta, method="classic")(da) + nsta, nlta = round(sta / 0.01), round(lta / 0.01) + for lane in range(da.sizes["distance"]): + expected = classic_sta_lta(da.values[:, lane], nsta, nlta) + np.testing.assert_allclose(result.values[:, lane], expected, rtol=1e-10) + # the same zero warm-up, sample for sample + np.testing.assert_array_equal(result.values[:, lane] == 0, expected == 0) + + @pytest.mark.parametrize("method", BOXCAR_METHODS) + @pytest.mark.parametrize("sta, lta", [(0.5, 10.0), (0.2, 3.0)]) + def test_boxcar_matches_direct_sums(self, method, sta, lta): + da = generate_stalta() + result = STALTA(sta, lta, method=method)(da) + nsta, nlta = round(sta / 0.01), round(lta / 0.01) + expected = boxcar_reference(da.values[:, 0], nsta, nlta, method) + np.testing.assert_allclose(result.values[:, 0], expected, rtol=1e-10) + + @pytest.mark.parametrize("method", STALTA_METHODS) + def test_dim_on_any_axis(self, method): + da = generate_stalta() + expected = STALTA(0.5, 10.0, method=method)(da) + result = STALTA(0.5, 10.0, method=method)(da.transpose("distance", "time")) + assert result.dims == ("distance", "time") + np.testing.assert_array_equal(result.values, expected.values.T) + + @pytest.mark.parametrize("method", STALTA_METHODS) + def test_chunk_invariant(self, method): + da = generate_stalta() + atom = STALTA(0.5, 10.0, method=method) + xd.testing.assert_chunk_invariant(atom, da, chunks={"time": 1000}, gaps=2) + # chunks far shorter than the long window, and than what "centered" + # holds back + xd.testing.assert_chunk_invariant(atom, da, chunks={"time": 37}, cuts=0) + + @pytest.mark.parametrize("method", STALTA_METHODS) + def test_chunked_along_another_dim(self, method): + da = generate_stalta() + expected = STALTA(0.5, 10.0, method=method)(da) + result = STALTA(0.5, 10.0, method=method).process(da, chunks={"distance": 1}) + np.testing.assert_array_equal(result.values, expected.values) + + @pytest.mark.parametrize("method", STALTA_METHODS) + def test_scale_invariant(self, method): + # DAS strain rates are ~1e-9: their energy must not hit the floor + da = generate_stalta() + expected = STALTA(0.5, 10.0, method=method)(da).values + result = STALTA(0.5, 10.0, method=method)(da * 1e-9).values + np.testing.assert_allclose(result, expected, rtol=1e-9) + + @pytest.mark.parametrize("method", BOXCAR_METHODS) + def test_round_off_does_not_outlive_a_loud_transient(self, method): + # A burst 1e9 times louder than what follows: a cumulative sum keeps + # its round-off for the rest of the trace, the running sums for at + # most two windows. + da = generate_stalta(nt=12000, nx=1) + data = da.values.copy() + data[2000:2100] *= 1e6 + data[3000:] *= 1e-3 + da = da.copy(data=data) + nsta, nlta = 50, 1000 + result = STALTA(0.5, 10.0, method=method)(da).values[:, 0] + expected = boxcar_reference(data[:, 0], nsta, nlta, method) + clean = slice(2100 + 2 * (nsta + nlta), None) + np.testing.assert_allclose(result[clean], expected[clean], rtol=1e-8) + + @pytest.mark.parametrize("method", ["recursive", "classic", "delayed"]) + def test_causal_methods_ignore_the_future(self, method): + da = generate_stalta() + expected = STALTA(0.5, 10.0, method=method)(da).values + data = da.values.copy() + data[2500:] = 100.0 + result = STALTA(0.5, 10.0, method=method)(da.copy(data=data)).values + np.testing.assert_array_equal(result[:2500], expected[:2500]) + assert not np.array_equal(result[2500:], expected[2500:]) + + @pytest.mark.parametrize("method", ["recursive", "classic", "delayed"]) + def test_causal_methods_hold_nothing_back(self, method): + da = generate_stalta() + atom = STALTA(0.5, 10.0, method=method) + for chunk in xd.split(da, 7, dim="time"): + out = atom(chunk, chunk_dim="time") + assert out.sizes["time"] == chunk.sizes["time"] + assert atom.flush() == [] + + def test_centered_holds_back_half_the_long_window(self): + da = generate_stalta() + atom = STALTA(0.5, 10.0, method="centered") + outs = [atom(chunk, chunk_dim="time") for chunk in xd.split(da, 4, dim="time")] + emitted = xd.concat(outs, "time").sizes["time"] + assert emitted == da.sizes["time"] - 1000 // 2 + (tail,) = atom.flush() + assert tail.sizes["time"] == 1000 // 2 + np.testing.assert_array_equal(tail.values, 0.0) + assert atom.flush() == [] + + def test_centered_onset_is_earlier_than_causal(self): + da = generate_stalta() + causal = STALTA(0.5, 10.0, method="classic")(da) + centered = STALTA(0.5, 10.0, method="centered")(da) + assert int(causal.values[:, 0].argmax()) > int(centered.values[:, 0].argmax()) + + @pytest.mark.parametrize("method", STALTA_METHODS) + def test_record_shorter_than_the_windows(self, method): + # nothing is ever complete: all zeros, chunked or not + da = generate_stalta(nt=300) + expected = STALTA(0.5, 10.0, method=method)(da) + np.testing.assert_array_equal(expected.values, 0.0) + result = STALTA(0.5, 10.0, method=method).process(da, chunks={"time": 40}) + np.testing.assert_array_equal(result.values, expected.values) + + def test_flush_without_data_is_empty(self): + # uninitialised, and initialised by an eager (unchunked) call: neither + # holds anything back + atom = STALTA(0.5, 10.0, method="centered") + assert atom.flush() == [] + atom(generate_stalta()) + assert atom.flush() == [] + + @pytest.mark.parametrize("method", STALTA_METHODS) + def test_parallel_matches_serial(self, method): + # each worker updates its own slice of lanes of the carried sums + da = generate_stalta(nx=7) + expected = STALTA(0.5, 10.0, method=method, parallel=False)(da) + result = STALTA(0.5, 10.0, method=method, parallel=3)(da) + np.testing.assert_array_equal(result.values, expected.values) + atom = STALTA(0.5, 10.0, method=method, parallel=3) + chunked = atom.process(da, chunks={"time": 2500}) + np.testing.assert_array_equal(chunked.values, expected.values) + + def test_twin_and_pipeline(self): + da = generate_stalta() + np.testing.assert_array_equal( + xd.stalta(da, sta=0.5, lta=10.0).values, STALTA(0.5, 10.0)(da).values + ) + assert isinstance(xd.stalta(..., sta=0.5, lta=10.0), STALTA) + picks = da >> (xd.stalta(..., sta=0.5, lta=10.0) >> xd.trigger(..., thresh=5.0)) + assert len(picks) > 0 + + def test_invalid_parameters(self): + with pytest.raises(ValueError, match="`method` must be one of"): + STALTA(0.5, 10.0, method="causal") + with pytest.raises(ValueError, match="strictly positive"): + STALTA(0.0, 1.0) + with pytest.raises(ValueError, match="longer than"): + STALTA(1.0, 0.5) + da = generate_stalta() + with pytest.raises(ValueError, match="shorter than one sample"): + STALTA(0.004, 10.0)(da) + with pytest.raises(ValueError, match="more samples than"): + STALTA(0.5, 0.504)(da) diff --git a/xdas/__init__.py b/xdas/__init__.py index 902c6a4a..718754df 100644 --- a/xdas/__init__.py +++ b/xdas/__init__.py @@ -72,6 +72,7 @@ "rechunk", "resample", "sliding_mean_removal", + "stalta", "stft", "taper", "trigger", @@ -97,7 +98,7 @@ # `sys.modules` and does not rebind the attribute: the lowercase twin below # stays `xdas.trigger` for everyone. from . import trigger as _trigger_module # noqa: F401 isort: skip -from .atoms.detect import trigger +from .atoms.detect import stalta, trigger from .atoms.kernel import rechunk from .atoms.ml import annotate, mlpicker, pick from .atoms.operations import ( diff --git a/xdas/atoms/__init__.py b/xdas/atoms/__init__.py index 093fa825..a4e9d869 100644 --- a/xdas/atoms/__init__.py +++ b/xdas/atoms/__init__.py @@ -18,6 +18,7 @@ """ __all__ = [ + "STALTA", "STFT", "Annotate", "Atom", @@ -43,11 +44,12 @@ "as_function", "atomized", "compose", + "stalta", "trigger", ] from .core import Atom, Partial, Sequential, State, as_function, atomized, compose -from .detect import Trigger, trigger +from .detect import STALTA, Trigger, stalta, trigger from .kernel import DownSample, LFilter, Polyphase, Rechunk, SOSFilter, UpSample from .ml import Annotate, MLPicker, Picker from .operations import ( diff --git a/xdas/atoms/detect.py b/xdas/atoms/detect.py index c8ae5287..0f2afeee 100644 --- a/xdas/atoms/detect.py +++ b/xdas/atoms/detect.py @@ -1,10 +1,11 @@ """ -Detection atoms: turn a characteristic function into picks. +Detection atoms: characteristic functions, and the picks drawn from them. These atoms emit :class:`pandas.DataFrame` objects rather than arrays — the transducer contract makes that unremarkable, a pick table is just another chunk type flowing downstream to a CSV sink. :class:`Trigger` is the threshold detector; its lowercase twin is :func:`xdas.trigger`. +:class:`STALTA` builds the characteristic function such a detector consumes. """ from collections.abc import Mapping @@ -13,11 +14,12 @@ import pandas as pd from numba import njit -from ..coordinates import Coordinate -from ..core import concat_coords +from ..coordinates import Coordinate, get_sampling_interval +from ..core import DataArray, concat, concat_coords +from ..parallel import get_workers_count, parallelize from .core import Atom, State, atomized -__all__ = ["Trigger", "trigger"] +__all__ = ["STALTA", "Trigger", "stalta", "trigger"] #: Name of the label dimension a mapping of thresholds keys on. It is the #: dimension :class:`~xdas.atoms.Annotate` appends to its characteristic @@ -594,3 +596,330 @@ def _trigger( # pragma: no cover trigger = atomized(Trigger) + + +class STALTA(Atom): + """ + Short-term over long-term average characteristic function. + + The ratio of a short mean of the squared signal to a long one: the classic + amplitude detector, whose output is fed to :class:`Trigger` to obtain + picks. Both window lengths are given in the physical units of `dim`, so + the same atom applies to any sampling rate. + + The ``recursive`` and ``classic`` methods follow their ObsPy counterparts + of :mod:`obspy.signal.trigger` sample for sample, including the samples + reported as zero while the windows are not yet full. All methods are + computed by running sums carried from chunk to chunk, so chunked processing + returns exactly what a single call on the whole record returns, at a cost + that does not depend on the window lengths. + + ============== ========================================================== + method definition + ============== ========================================================== + ``recursive`` Exponential averages of time constants `sta` and `lta` + (Evans and Allen), as ObsPy's ``recursive_sta_lta``. + The default, and the cheapest to stream: its state is two + numbers per channel. + ``classic`` Trailing boxcar means, as ObsPy's ``classic_sta_lta``. + ``delayed`` Trailing boxcar means, the long window ending where the + short one starts so that a transient does not raise its + own reference (Withers et al., 1998). + ``centered`` Boxcar means centred on the sample they produce, so the + peak sits on the transient, but each window reaches `lta` / + 2 into the future: + offline only, the samples it cannot yet complete being + held back until :meth:`flush`. + ============== ========================================================== + + The choice moves pick times, so it is a property of the detector, not a + tuning knob: the peak of a trailing boxcar lags the transient by about half + the short window, that of ``centered`` does not. + + Parameters + ---------- + sta : float + Length of the short-term window, in the units of `dim`. + lta : float + Length of the long-term window, in the units of `dim`. Must be longer + than `sta`. + method : {"recursive", "classic", "delayed", "centered"}, optional + How the two averages are computed, by default "recursive". + dim : str, optional + The dimension along which to compute the ratio, by default "time". + parallel : int, bool, or None, optional + Worker count for parallelisation across channels. + + Notes + ----- + Samples whose windows are incomplete — the start of each run, and with + ``"centered"`` its end too — are zero, which no positive threshold + triggers on. The boxcar sums are recomputed from scratch once per window, + so the round-off of a loud transient does not outlive two windows (the + cumulative sum of ObsPy's ``classic_sta_lta`` keeps it for the rest of the + trace). ObsPy's ``delayed_sta_lta`` is not used as a reference: as of + ObsPy 1.5 it adds the samples leaving its windows instead of subtracting + them. + + References + ---------- + Withers, M., Aster, R., Young, C., Beiriger, J., Harris, M., Moore, S., + and Trujillo, J. (1998). A comparison of select trigger algorithms for + automated global seismic phase and event detection. Bulletin of the + Seismological Society of America, 88(1), 95-106. + + Examples + -------- + >>> import numpy as np + >>> import xdas as xd + + A transient buried in noise: + + >>> rng = np.random.default_rng(42) + >>> data = rng.normal(0.0, 1.0, 2000) + >>> data[1200:1240] += 20.0 + >>> da = xd.DataArray( + ... data=data, + ... coords={"time": {"tie_indices": [0, 1999], "tie_values": [0.0, 199.9], + ... "sampling_interval": 0.1}}, + ... ) + + Eager processing with the functional twin: + + >>> cft = xd.stalta(da, sta=1.0, lta=20.0) + >>> float(cft.max().values) > 10.0 + True + + Chunked processing gives the same answer, seam for seam: + + >>> atom = xd.stalta(..., sta=1.0, lta=20.0) + >>> chunked = atom.process(da, chunks={"time": 250}) + >>> bool(np.array_equal(chunked.values, cft.values)) + True + + Piped into the trigger, which is the point of it: + + >>> picks = xd.stalta(da, sta=1.0, lta=20.0) >> xd.trigger(..., thresh=8.0) + + """ + + def __init__(self, sta, lta, method="recursive", dim="time", parallel=None): + super().__init__() + if method not in _STALTA_METHODS: + raise ValueError( + f"`method` must be one of {', '.join(map(repr, _STALTA_METHODS))}" + ) + if not sta > 0.0: + raise ValueError("`sta` must be strictly positive") + if not lta > sta: + raise ValueError("`lta` must be longer than `sta`") + self.sta = float(sta) + self.lta = float(lta) + self.method = method + self.dim = dim + self.parallel = parallel + + # states + self.delta = State(...) + self.count = State(...) + self.sums = State(...) + self.ring = State(...) + self.buffer = State(...) + + def initialize(self, da, chunk_dim=None, **flags): + """Measure the sampling interval to size the windows, and zero the sums.""" + self.delta = State(float(get_sampling_interval(da, self.dim))) + self.initialize_from_state() + lanes = da.size // da.sizes[self.dim] + self.count = State(0) + self.sums = State(self._fresh_sums(lanes)) + self.ring = State(np.zeros((lanes, self.history))) + # `None` marks a whole record per call (no seam along `dim` to cross) + if chunk_dim == self.dim: + self.buffer = State(da.isel({self.dim: slice(0, 0)})) + else: + self.buffer = State(None) + + def initialize_from_state(self): + """Derive the window lengths and offsets, in samples.""" + self.nsta = round(self.sta / self.delta) + self.nlta = round(self.lta / self.delta) + if self.nsta < 1: + raise ValueError("`sta` is shorter than one sample") + if not self.nlta > self.nsta: + raise ValueError("`lta` must span more samples than `sta`") + # each boxcar window ends `delay` samples before the newest one + self.lag = 0 + match self.method: + case "delayed": + self.dsta, self.dlta = 0, self.nsta + case "centered": + self.lag = self.nlta // 2 + self.dsta, self.dlta = self.lag - self.nsta // 2, 0 + case _: + self.dsta = self.dlta = 0 + if self.method == "recursive": + self.start, self.history = self.nlta, 0 + else: + reach = max(self.nsta + self.dsta, self.nlta + self.dlta) + self.start, self.history = reach - 1, reach + 1 + + def call(self, da, **flags): + """Emit the ratio for every sample the stream so far determines.""" + if self.buffer is None: # whole record: start afresh, end with the tail + lanes = da.size // da.sizes[self.dim] + self.count = State(0) + self.sums = State(self._fresh_sums(lanes)) + self.ring = State(np.zeros((lanes, self.history))) + shifted = self._advance(da)[..., self.lag :] + values = np.zeros(shifted.shape[:-1] + (da.sizes[self.dim],)) + values[..., : shifted.shape[-1]] = shifted + return da.copy(data=self._unlane(values, da)) + values = self._advance(da) + x = concat([self.buffer, da], self.dim) if self.buffer.sizes[self.dim] else da + # the newest `lag` samples stay held back: their windows reach samples + # yet to come + emit = max(x.sizes[self.dim] - self.lag, 0) + self.buffer = State(x.isel({self.dim: slice(emit, None)})) + if not emit: + return [] + values = values[..., values.shape[-1] - emit :] + out = x.isel({self.dim: slice(0, emit)}) + return out.copy(data=self._unlane(values, out)) + + def flush(self): + """Release the held-back tail, zero since its windows run off the record.""" + # Not `self.buffer is None`: a flush also reaches an atom that was + # never chunked (buffer `None`) or never called at all (buffer `...`). + if not isinstance(self.buffer, DataArray) or not self.buffer.sizes[self.dim]: + return [] + tail = self.buffer.copy(data=np.zeros(self.buffer.shape)) + self.buffer = State(self.buffer.isel({self.dim: slice(0, 0)})) + return [tail] + + def _fresh_sums(self, lanes): + """Return the initial running sums, two per lane, ObsPy's for "recursive".""" + sums = np.zeros((lanes, 2)) + if self.method == "recursive": + sums[:, 1] = _TINY + return sums + + def _advance(self, da): + """Feed one block to the kernel, returning its ratios lane by lane.""" + values = np.moveaxis(np.asarray(da.values), da.get_axis_num(self.dim), -1) + # C order: the kernels walk each lane along `dim`, which `moveaxis` strides + energy = np.square(values.reshape(-1, values.shape[-1]), dtype=float, order="C") + # the kernels update the sums and the ring in place, each worker its + # own slice of lanes; the pool is sized by the samples processed, not + # by the ring, which a call barely touches + parallel = get_workers_count(self.parallel, energy.nbytes) + if self.method == "recursive": + func = parallelize((0, 0, None, None, None), 0, parallel)(_recursive_ratio) + out = func(energy, self.sums, self.count, self.nsta, self.nlta) + else: + func = parallelize((0, 0, 0) + (None,) * 6, 0, parallel)(_boxcar_ratio) + out = func( + energy, + self.sums, + self.ring, + self.count, + self.nsta, + self.dsta, + self.nlta, + self.dlta, + self.start, + ) + self.count = State(self.count + energy.shape[-1]) + return out + + def _unlane(self, values, like): + """Fold lane-by-lane ratios back into the layout of `like`.""" + axis = like.get_axis_num(self.dim) + shape = np.moveaxis(np.empty(like.shape, dtype=bool), axis, -1).shape + return np.moveaxis(values.reshape(shape), -1, axis) + + +_STALTA_METHODS = ("recursive", "classic", "delayed", "centered") + +#: The floor of the long-term average, ObsPy's: smallest normal float64. +_TINY = np.finfo(np.float64).tiny + + +@njit(nogil=True, cache=True) +def _recursive_ratio(energy, sums, count, nsta, nlta): # pragma: no cover + """ + Evans and Allen recursive STA/LTA, resumable, as ObsPy's. + + `sums` holds the two averages of each lane and is updated in place; + `count` is the index, in the run, of the first sample of `energy`. + """ + csta = 1.0 / nsta + clta = 1.0 / nlta + icsta = 1.0 - csta + iclta = 1.0 - clta + lanes, size = energy.shape + out = np.zeros((lanes, size)) + for lane in range(lanes): + sta = sums[lane, 0] + lta = sums[lane, 1] + for j in range(size): + t = count + j + if t == 0: # ObsPy's recursion starts at the second sample + continue + sta = csta * energy[lane, j] + icsta * sta + lta = clta * energy[lane, j] + iclta * lta + if t >= nlta: + out[lane, j] = sta / max(lta, _TINY) + sums[lane, 0] = sta + sums[lane, 1] = lta + return out + + +@njit(nogil=True, cache=True) +def _boxcar_ratio( + energy, sums, ring, count, nsta, dsta, nlta, dlta, start +): # pragma: no cover + """ + Ratio of two boxcar means, resumable. + + Each window spans `n` samples and ends `d` samples before the newest one. + `ring` keeps the last ``ring.shape[1]`` energies of each lane and `sums` + the two running sums, both updated in place; `count` is the index, in the + run, of the first sample of `energy`. Samples before `start` are zero. + """ + lanes, size = energy.shape + out = np.zeros((lanes, size)) + for lane in range(lanes): + ssta = sums[lane, 0] + slta = sums[lane, 1] + for j in range(size): + t = count + j + ring[lane, t % ring.shape[1]] = energy[lane, j] + ssta = _slide(ring[lane], ssta, t - dsta, nsta) + slta = _slide(ring[lane], slta, t - dlta, nlta) + if t >= start: + out[lane, j] = (ssta / nsta) / max(slta / nlta, _TINY) + sums[lane, 0] = ssta + sums[lane, 1] = slta + return out + + +@njit(nogil=True, cache=True) +def _slide(ring, total, last, n): # pragma: no cover + """Move a running sum of `n` samples to the window ending at `last`.""" + if last < 0: + return 0.0 + size = ring.shape[0] + # once per window, sum from scratch: what sliding accumulated is dropped + if (last + 1) % n == 0: + total = 0.0 + for k in range(last - n + 1, last + 1): + total += ring[k % size] + return total + total += ring[last % size] + if last >= n: + total -= ring[(last - n) % size] + return total + + +stalta = atomized(STALTA)