Repository navigation
Re-measure MATLAB vs Python speed (~7x) and fix benchmark_express for express bundles - #171
Merged
Merged
Conversation
… for bundles - PIPELINE_PORT_STATUS.md: new top-of-section comparison on the same ExampleData benchmark. MATLAB 893 s vs Python 125.6 s full (~7x; 105 s in a quieter run, 8.5x), express 53 s. The July 2.66x stays below as history. - MEApipeline_timingBenchmark.m: record the re-run and how to launch it headless. - benchmark_express.py: express mode now replaces the output folder with a .meanap bundle, so read step timings from the log and measure the bundle instead of crashing on iterdir(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LBjNAqNrYVFQSwnaKgi25j
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-measured the documented MATLAB-vs-Python benchmark after #170, and fixed the benchmark script, which crashed on express runs.
Speed vs MATLAB
Same dataset and settings as the July comparison:
ExampleDataA2/A3, Axion64, lags 10/25/50 ms, 200 shuffles, all figures. MATLAB R2025b ran with a 16-worker parpool, and Python ran straight afterwards on the same machine.Two quieter Python runs took 105.6 s and 105.3 s (8.5x), so 7–8x is the range. Express mode takes 53 s. Against the code before #170 (
356fc0f), all CSVs, adjacency matrices and spike files are byte-identical, in both full and express mode.Changes
python/PIPELINE_PORT_STATUS.md: the new comparison goes at the top of the MATLAB-vs-Python section; the July 2.66x stays below as history.python/parityRun/MEApipeline_timingBenchmark.m: records the re-run and the headless launch command.python/benchmark_express.py: express mode replaces the output folder with a.meanapbundle, andstep_durations.jsonis deleted along with the folder. The script crashed oniterdir()of the bundle; it now reads step timings from the run log and sizes the bundle's contents. Verified with a full end-to-end run (exit 0; 105.3 s full / 53.2 s express).Machine caveat
Measured on an AMD Ryzen AI Max+ 395 (16 physical cores, 109 GB RAM). Both pipelines scale with core count, and most of Python's recent gain comes from using more cores, so the ratio is expected to be smaller on an 8-core laptop. The doc now says so and explains how to re-run both sides on another machine. A run on a machine with fewer cores is planned.
🤖 Generated with Claude Code
https://claude.ai/code/session_01LBjNAqNrYVFQSwnaKgi25j