Repository navigation
Electrophysiology rig formats and BIDS sidecars (0.10.0) - #10
Merged
Merged
Conversation
Six decoders for what an EEG, iEEG or Neuropixels rig writes beside its samples, and for the sidecars that describe a BIDS dataset. - EDF/EDF+/BDF/BDF+: flavour, bit depth, start time resolved from the EDF+ recording field, patient and recording subfields, record count and duration, and per signal label, units, ranges, rate, transducer and prefiltering. Checked against pyedflib. - BrainVision .vhdr and .vmrk: format, orientation, channel table, sampling rate, duration, markers by type and the segment timestamp. Checked against mne on all 64 non-empty .vhdr files in bids-standard/bids-examples. - SpikeGLX .meta: the only description a headerless Neuropixels binary has. Rate, AP/LF/sync counts, run name with gate and trigger, probe identity, derived sample count and microvolts per bit. Checked against neo. - Neuralynx .ncs/.nev/.nse/.ntt/.nvt: the 16 KiB ASCII header in both dialects. Checked against python-neo's header grammar. - Open Ephys structure.oebin: streams, rates, channel counts and scaling. - BIDS sidecars under .json and .tsv, content-sniffed so neither generic extension is claimed without a key or column signature the standard defines. Swept over the whole bids-examples corpus. Registering .json and .tsv means dispatch now reaches a decoder for those extensions; a name-registered decoder still wins and non-BIDS content is declined, which is what tests/test_zarr.py now asserts.
pyedflib refuses to open a file whose samples are absent and PhysioNet truncated every attempt to download all 48 MB of SC4001E0-PSG.edf, so that case rests on the header's own redundancy, not on a reference reader. The other five were checked against pyedflib.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Six new decoders: what an EEG, iEEG or Neuropixels rig writes beside its raw samples, plus the sidecars that describe a dataset organised to the Brain Imaging Data Structure. These are the formats BIDS wraps, so a pipeline that only reads
.niiand.nwbsees a session it cannot describe..edf,.bdf.vhdr,.vmrk.vhdrfiles in bids-standard/bids-examples; all 64 agreed.meta.ncs,.nev,.nse,.ntt,.nvtstructure.oebin.json,.tsvNotes worth a reviewer's attention:
.jsonand.tsvare now registered extensions. Both are maximally generic, so the BIDS decoders decline anything without a key the standard defines or one of its column signatures. A name-registered decoder (zarr.json,.zattrs,attributes.json) still wins over the extension.tests/test_zarr.pypreviously asserted that no decoder claimed a generic.jsonname; it now asserts the stronger thing, that unrelated JSON content decodes to nothing.RepetitionTime, notrepetitionTime). These are the names the standard, its validator and downstream tools use, and recasing them would make a record impossible to match against the specification.Golden fixtures follow the existing contract (hex in, expected record out) so the Go port can load the same files. Suite is 587 passed, 2 skipped, green on Python 3.9 and 3.12 and on an interpreter with nothing but pytest installed.