Processing Pipeline

Processing Pipeline

This page covers the core processFNIRS2 pipeline: how raw light intensity becomes hemoglobin concentrations, the three processing stages, DPF (pathlength) modes, the programmatic Pipeline API for building custom method chains, and pf2.ProcessingContext for reproducible/parallel processing. It is for anyone moving from an imported recording to processed HbO/HbR data.

The three-stage pipeline

Every call to processFNIRS2 moves data through three stages:

Stage	Function	Input → Output
1. Raw → Optical Density	pf2_base.fnirs.processStageRaw2OD	Raw light intensity → optical density (motion correction, filtering, ambient subtraction, ICA, CAR — methods configured in +pf2/+methods/+raw)
2. OD → Hemoglobin	pf2_base.fnirs.processStageOD2Hb (applies the modified Beer-Lambert law via pf2_base.fnirs.bvoxy)	Optical density → HbO, HbR, HbTotal, HbDiff, CBSI
3. Hemoglobin filtering	pf2_base.fnirs.processStageFilterHb	Hemoglobin → filtered hemoglobin, ROI averages (methods configured in +pf2/+methods/+oxy)

Some Stage 1 motion-correction methods (TDDR, Spline, Wavelet, MARA) require optical density as input (requiresOD in pf2_functions_default.cfg); the raw stage validates this and errors if such a method is placed before the OD conversion step in a chain.

Run pf2.methods.raw.list() and pf2.methods.oxy.list() for the live set of named methods on your installation. Two raw seeds ship by default: OD_TDDR (log→OD, then TDDR motion correction) and OD_SMAR (log→OD, then SMAR sliding-window rejection). Common oxy-stage methods include LPF, HPF, BPF (0.01–0.1 Hz Butterworth), LPF_ROI, and takizawa_easy/takizawa_hard (automatic channel rejection).

Calling processFNIRS2

data      = pf2.import.sampleData.fNIR2000();      % sample recording, no files needed
processFNIRS2(data);                                % opens the GUI (no output captured)
processed = processFNIRS2(data);                    % headless — output capture suppresses the GUI
processed = processFNIRS2(data, 'ShowGUI', true);   % force the GUI even with an output captured
processed = processFNIRS2(data, 'OD_TDDR', 'BPF');  % named raw/oxy methods (positional)
allData   = processFNIRS2(allData);                 % cell array: process each element

GUI rule: assigning an output (processed = processFNIRS2(...)) automatically suppresses the GUI. Never pass 'ShowGUI', false together with a captured output — it is redundant. Use 'ShowGUI', true only when you want the GUI and an output.

Cell-array batches: passing a cell array processes every element and returns a cell array of results. For more than two elements with a parallel pool running, processFNIRS2 automatically uses parfor (pass 'Accelerate', 'none' to opt out).

Noop pass-through: an element with no .raw light intensities has nothing to convert, so processFNIRS2 returns it unchanged instead of demanding a device/probe config. This covers empty cells ([]) and data already in biomarker form (e.g. from pf2.import.fromTable or a prior processing run), letting a batch mix raw recordings with already-processed or empty elements:

out = processFNIRS2({rawRecording, [], alreadyProcessedHb});  % processes only the first

The noop only applies when the GUI is suppressed (an output is captured, or the element is inside a cell array); a bare processFNIRS2(hbStruct) with no output still opens the GUI per the rule above. SkipRaw (oxy-stage-only processing) and SkipOD (stop after the raw stage) are exempt and always follow the normal path.

odOnly  = processFNIRS2(data, 'SkipOD', true);    % stop after Stage 1, return optical density
oxyOnly = processFNIRS2(odData, 'SkipRaw', true); % skip Stage 1, run only the oxy stage on existing Hb/OD

DPF modes

Stage 2 (OD → Hb) needs a pathlength assumption. Set it with 'DPFmode':

Mode	Description
'None'	No DPF correction; units are mM·mm
'Fixed'	Single DPF value applied to all channels (default 5.93)
'Calc'	Age/wavelength-dependent DPF (Scholkmann & Wolf, 2013)
'PPF'	Escape hatch: supply a complete partial pathlength factor directly (L = SD .* ppf, MNE/Homer convention)

processed = processFNIRS2(data, 'DPFmode', 'Calc', 'defaultSubjectAge', 30);
processed = processFNIRS2(data, 'DPFmode', 'Fixed', 'FixedDPF', 5.93);
processed = processFNIRS2(data, 'DPFmode', 'None');   % units in mM*mm

'Calc' also accepts a partial-volume correction divisor via 'PVC' (a numeric value ≥ 1, or 'auto' to derive it per channel from source-detector separation via pf2_base.fnirs.strangmanPVC).

Common processing parameters

Parameter	Typical value	Purpose
blLength	5–10 sec	Baseline duration
blStartTime	0 sec	Baseline start relative to t0
defaultSubjectAge	25 years	For DPF calculation
DPFmode	'Calc'	Age-dependent DPF
resample rate	2–10 Hz	Common time grid for multi-device work
LPF cutoff	0.1 Hz	Remove cardiac/respiratory frequencies
BPF range	0.008–0.1 Hz	Isolate the hemodynamic response
SCI threshold	0.75	Scalp coupling index channel rejection

Building custom pipelines with the Pipeline API

The Pipeline class hierarchy is a chainable, value-semantics alternative to hand-editing method structs. pf2_base.RawPipeline builds Stage 1 chains; pf2_base.OxyPipeline builds Stage 3 chains (and adds ROI helpers).

% Build a raw pipeline: OD conversion + TDDR motion correction
raw = pf2_base.RawPipeline('myRawPipeline');
raw = raw.add('pf2_Intensity2OD');
raw = raw.add('pf2_MotionCorrectTDDR');
raw = raw.add('pf2_lpf', 'freq_cut', 0.2);          % name-value overrides on add()

% Build an oxy pipeline: low-pass filter + ROI averaging
oxy = pf2_base.OxyPipeline('myOxyPipeline');
oxy = oxy.add('pf2_lpf', 'freq_cut', 0.08, 'Nf', 80);
oxy = oxy.add('pf2_build_nanmean_ROI');
oxy2 = oxy.swapROI('pf2_build_pca_ROI', 'ComponentNumber', 2);  % swap/append the ROI step

% Inspect and tune
tbl = oxy.params();                                  % table of all tunable parameters
oxy = oxy.setParam('pf2_lpf', 'freq_cut', 0.12);      % single parameter
oxy = oxy.setParams('pf2_lpf', 'freq_cut', 0.1, 'Nf', 50);  % bulk update

% Restructure (all methods use value semantics — originals are unchanged)
p = pf2_base.OxyPipeline('modify_demo');
p = p.add('detrend');
p = p.add('pf2_lpf', 'freq_cut', 0.1);
p = p.insert(2, 'pf2_hpf', 'freq_cut', 0.01);   % insert at a 1-based index
p = p.remove('detrend');                         % remove by function name
p = p.swapStep('pf2_hpf', 'pf2_bpf_butter', 'lowF', 0.01, 'highF', 0.08);

% Save and run
raw.save('raw', 'Replace', true);   % register as a named raw method
oxy.save('oxy', 'Replace', true);   % register as a named oxy method
data      = pf2.import.sampleData.fNIR2000();
processed = processFNIRS2(data, 'myRawPipeline', 'myOxyPipeline');

% Reload a saved pipeline back into the class form
rawLoaded = pf2_base.RawPipeline.fromMethod('myRawPipeline');

% Convert to/from the legacy method struct
m = raw.toMethod();

See examples/scripts/example_pipeline_basics.m for the full walkthrough (construction, inspection, structural edits, save/run/round-trip).

Registering custom functions

Any MATLAB function can join a pipeline once it is registered. PipelineFunction.detect auto-discovers a function's signature (from pf2_functions_default.cfg if present, otherwise by parsing the source file), and register persists it so Pipeline.add() and the GUI can find it:

pf = pf2_base.PipelineFunction.detect('myCustomFunction');
pf2_base.PipelineFunction.register(pf);

raw = pf2_base.RawPipeline('customRaw');
raw = raw.add('pf2_Intensity2OD');
raw = raw.add('myCustomFunction');   % now discoverable after registration
raw = raw.add('pf2_MotionCorrectTDDR');

A pipeline can also be built directly from MATLAB call-syntax strings:

p = pf2_base.Pipeline('from_strings');
p = p.addFromString('pf2_Intensity2OD(x)');
p = p.addFromString('[x]=pf2_MotionCorrectTDDR(x,fs)');
p = p.addFromString('[x]=pf2_lpf(x,1,fs,0.15,80)');

See examples/scripts/example_pipeline_custom_function.m for the detect → register → use → clean-up cycle.

Reproducible and parallel processing with ProcessingContext

pf2.ProcessingContext bundles every setting processFNIRS2 needs — DPF mode, baseline, methods, device, rejection level — into an object you configure explicitly, instead of relying on the global PF2/setF state. Passing it via 'Context' makes processing fully isolated: globals are neither read nor written on that path, which makes it safe for parfor and byte-for-byte reproducible.

% Configure in one call — usable immediately, no fromGlobals() bootstrap needed
ctx = pf2.ProcessingContext('DPFmode', 'Calc', 'SubjectAge', 30, 'RawMethod', 'OD_TDDR');
result = ctx.process(data);                       % equivalent to processFNIRS2(data, 'Context', ctx)

ctx.configure('blLength', 5);                     % chainable Name-Value setter (mutates in place)

% Parallel: give each worker an independent copy (note copy(), not '=', which aliases a handle)
parfor i = 1:numel(allData)
    c = ctx.copy();
    c.subjectAge = ages(i);
    results{i} = processFNIRS2(allData{i}, 'Context', c);
end

The context is one-directional: it never writes settings back to the globals. pf2_base.ProcessingContext.fromGlobals() remains available to snapshot the current global/GUI state into a context (used internally when a cell-array batch auto-parallelizes), but ordinary reproducible work should build a pf2.ProcessingContext explicitly rather than depend on whatever the globals currently hold.

The processed output structure

processed.HbO / .HbR / .HbTotal / .HbDiff / .CBSI   % [T x C] biomarkers
processed.units          % units of the biomarker data (µM, or mM*mm under DPFmode 'None')
processed.DPF_factor     % DPF value(s) used in the conversion
processed.device         % pf2.Device object (immutable device config)
processed.processingInfo % settings used, for reproducibility and plot annotation

processingInfo records timestamp, dpfMode, dpfValue, subjectAge, baselineStart, baselineLength, rawMethod, oxyMethod, deviceName, samplingRate, and rejectLevel — enough to reproduce the run or annotate a plot.

See also

• Importing Data
• Block Averaging and Epoching
• Quality Control
• Group Analysis