Repository navigation
Add STALTA characteristic-function atom - #86
Conversation
… methods
Replace mode/window/pad_mode with method={recursive (default), classic,
delayed, centered}. Running sums in numba kernels stream at O(1) per sample
(was a full FFT convolution of the buffer per chunk, 54x slower than eager on
1 s chunks), floor the long-term average at tiny instead of eps (the ratio
collapsed on ~1e-9 strain rates), zero the warm-up as ObsPy does instead of
reflect-padding the record start (which leaked the future into causal output),
and re-sum each boxcar once per window so a loud transient's round-off does
not outlive two windows.
recursive is bit-exact against obspy.signal.trigger.recursive_sta_lta, classic
within 1e-10 of classic_sta_lta; delayed and centered are checked against
direct window sums (ObsPy 1.5's delayed_sta_lta adds instead of subtracts).
Add parallel= like the other kernel atoms: nogil numba kernels split by lane through xdas.parallel.parallelize, the pool sized by the samples processed rather than by the carried ring (which a streamed chunk barely touches). Lay the energy out in C order: on time-first data moveaxis left it strided and the kernels missed cache on every sample (classic 6x slower).
atrabattoni
left a comment
There was a problem hiding this comment.
Hi @amaggi, thanks for this, STA/LTA feeding straight into Trigger is really useful.
I compared it with ObsPy and ran into a few issues, so I pushed three commits on top of yours:
- the LTA is floored at
eps, which is too high for DAS data in strain rate (values around 1e-9): the ratio drops to ~0.004 - after a strong event, the
fftconvolveround-off gives absurd ratios in the quiet part that follows - in causal mode, the
reflectpadding at the start uses future samples - in streaming, each chunk re-convolves the whole buffer, so it gets very slow with small chunks (54x slower than one call on 1 s chunks)
I replaced mode/window/pad_mode with a method argument that follows ObsPy: "recursive" (now the default, same values as recursive_sta_lta), "classic" (matches classic_sta_lta), "delayed", and I kept your "centered" mode for offline use. Under the hood it's now running sums in numba, so the cost doesn't depend on the window lengths anymore and chunked results are identical to a single call. I also added parallel= like the other filters, and tests against ObsPy.
Side note: ObsPy's delayed_sta_lta looks buggy (it adds the leaving samples instead of subtracting them), so I tested that one against plain window sums.
These changes shouldn't affect your code, so I'll merge it as is.
Pushes to a fork fire the workflows in the fork, where Actions are off by default, so pull requests from forks were never tested. Trigger on pull_request too, and limit push to main and dev to avoid running twice for branches pushed to origin.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #86 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 49 49
Lines 9293 9391 +98
Branches 1680 1694 +14
=========================================
+ Hits 9293 9391 +98 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
On macOS arm64 the compiler fuses the recursion's multiply-add in one of the two builds, so values differ in the last bits (relative 1e-15) and the bit-exact comparison failed on CI.
Add an
STALTAcharacteristic-function atomXdas has a threshold detector (
Trigger) but nothing to feed it apart from aSeisBench model. This adds the classic amplitude detector — the ratio of a
short-term to a long-term average of the squared signal — as an atom, so
detection no longer needs a network of weights:
or, as a pipeline that streams:
What it does
staandltaare given in the units ofdim, and the sample counts are derived ininitializefrom the measuredsampling rate, the way
operations.Filterdoes — so a parameter survives achange of rate.
mode="causal"(default) averages the samples up to andincluding the one it produces, as a real-time detector must;
mode="centered"averages symmetrically about it, which sharpens the onset but reaches into the
future and so is offline only. The choice moves pick times, so it is a
property of the detector rather than a tuning knob.
windowandpad_modefor the taper applied to both means and for howthe two ends of the record are extended. Chunk seams are never padded: they
use the real neighbouring samples.
chunked run returns exactly what a single call on the whole record returns —
tested over 2, 5, 17 and 200 chunks in both modes, and for a record shorter
than the halo. In
"centered"mode the samples whose window is not yetcomplete are held back and
flush()releases them; in"causal"mode nothingis held back and
flush()has nothing to return.Notes for review
Trigger. It already turns a characteristic functioninto picks, statefully, and does it well; this only supplies another kind of
input to it.
recursive_sta_ltais deliberately left out. The recursive form is adifferent (and genuinely streaming-native) estimator, and belongs in its own
PR rather than as a
method=flag bolted on here.STALTAdoes not survivesave_state/load_state. Its buffer is aDataArray, andAtom.set_stateconverts aDataArraystate to a barendarray(xdas/atoms/core.py, the# TODO: shouldn't need __array__line),so the coordinates are lost. This is the same limitation
DownSample,RechunkandPolyphasealready have — the two commented-out round-triptests in
tests/test_atoms.pyare about exactly this — so I have not workedaround it here. Happy to take it on as a follow-up if you want the state
machinery finished.
Checks
ruff checkandruff formatclean.xdas/atoms/detect.pyat 100% coverage bar one pre-existing line(
Trigger.merge's empty-table branch, covered bytest_atoms_ml.py).reproduce on a clean
devon macOS arm64:tests/test_signal.py::TestSTFT::test_parrallelandtests/virtual/test_tiles.py::TestStoredForm::test_non_utf8_root_survives_the_headerfail on
devas they do here.tests/test_picking.pyruns in the sameprocess as anything that has imported torch (
tests/test_atoms_ml.py, butalso
dascore/seisbenchpulled in elsewhere) — torch's OpenMP againstnumba's
@njit(parallel=True)inxdas/picking.py. NeitherKMP_DUPLICATE_LIB_OK=TRUEnorNUMBA_NUM_THREADS=1helps. Runningtests/test_picking.pyin its own process is the workaround; worth aseparate issue.