[aaronkanzer]

Hardening of the FCS decoder, prompted by the FlowIO delimiter work (whitews/FlowIO#46), where the maintainer flagged that legacy FCS 2.0 / 3.0 vendor files do not follow the escaping rule.

Legacy empty values. BD CellQuest and other old writers put an empty value down as a doubled delimiter (KEY\\NEXT\value\). The splitter read that as an escaped literal, so KEY's value swallowed NEXT and every later pair swapped keyword and value. It now decides by position: a keyword or value may not begin with the delimiter (FCS 3.1 section 3.2.7), so a doubled delimiter straight after a keyword is an empty value, and inside a value it stays a literal ($P3F/488//10/ is 488/10, 561////10 nm is 561//10 nm, a run of three ends a value in a literal). flowCore's default emptyValue=TRUE makes the same call on keywords. The one layout this cannot read is a keyword that itself contains the delimiter, and that costs only that pair.

Also

• A supplemental TEXT segment inside the leading read is parsed, so $CYT and $PnS placed there are found. Primary TEXT wins. Writers that start it with the delimiter and writers that do not are both handled.
• A channel without $PnN takes its $PnS; when $PAR is missing or unusable, channels are listed from the $PnN keywords present.
• Values that are not UTF-8 (CellQuest's \xaa) decode as Latin-1 instead of replacement characters.
• $DATATYPE and $CYT are stripped, $DATATYPE upper-cased; a HEADER offset with a stray byte after the digits is read; multipleDatasets is set when $NEXTDATA points at a second dataset (Coulter LMD).
• The splitter works on bytes.split runs instead of a per-byte loop: 1 MiB of TEXT in 13 ms instead of about 300 ms.

Evidence

• 43 real FCS/LMD files from the flowio, fcsparser and FlowKit test suites plus 33 hand-built delimiter cases: version, $PAR, $TOT, channel names and $CYT all match a reference reader before and after.
• Every TEXT keyword compared with flowio 1.4.0 on the 32 corpus files flowio can open: identical except flowio's own data1.fcs, where flowio misreads the 4 empty CellQuest pairs and this now reads them.
• 1300 spec-valid mutants of the corpus (12 delimiters, escaped delimiters inside and at the end of values, $ in values, byte order, 24-bit widths): counts and channel names right on all 1300; the only keyword differences are the 4 corpus files whose re-serialized keywords contain the delimiter.
• Tests: 21 to 36 in test_fcs.py, including a seeded round-trip property test of the splitter. Full suite 495 passed on 3.14; FCS tests pass on 3.10 and 3.12.