Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 2 additions & 13 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -6,24 +6,13 @@
.bzrignore export-ignore

# pyamplicol tests and development tooling stay in the repository for its own
# PyPI releases. The MadGraph7 installer builds the wheel without them, except
# for the legacy-oracle modules that the wheel bundles.
# PyPI releases. The MadGraph7 installer builds the wheel without them.
pyamplicol/tests/ export-ignore
pyamplicol/tools/developer/** export-ignore
pyamplicol/tools/developer/legacy_amplicol.py -export-ignore
pyamplicol/tools/developer/legacy_oracle -export-ignore
pyamplicol/tools/developer/legacy_oracle/** -export-ignore
pyamplicol/tools/fft_profiling/ export-ignore
pyamplicol/umami_example/ export-ignore
pyamplicol/flake.nix export-ignore
pyamplicol/flake.lock export-ignore
pyamplicol/run_fft_profiling_render.sh export-ignore
pyamplicol/tools/developer/ export-ignore

# Generated pyamplicol data: GitHub collapses these in PR diffs.
pyamplicol/Cargo.lock linguist-generated
pyamplicol/flake.lock linguist-generated
pyamplicol/dependencies/python-runtime-lock.toml linguist-generated
pyamplicol/tests/fixtures/** linguist-generated
pyamplicol/src/pyamplicol/assets/selftest/** linguist-generated
pyamplicol/src/pyamplicol/assets/models/json/** linguist-generated
pyamplicol/src/pyamplicol/_profiling_campaign/results/*.json linguist-generated
8 changes: 1 addition & 7 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,7 @@ def _copy_pyamplicol_docs():
"_config.yml", "_includes", "css", "results", "archive", "arxiv", ".*"
)

def ignore(directory, names):
# Only the report index is tracked; campaigns are local output.
if os.path.basename(directory) == "performance_reports":
return {name for name in names if name != "README.md"}
return jekyll_only(directory, names)

shutil.copytree(source, target, ignore=ignore)
shutil.copytree(source, target, ignore=jekyll_only)


_copy_pyamplicol_docs()
Expand Down
141 changes: 2 additions & 139 deletions pyamplicol/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,152 +228,15 @@ The public C ABI is version 1. Every generated artifact can include standalone
Python, C11, C++17, Fortran 2008, and dependency-free Rust 2021 drivers backed
by the wheel-owned static Rusticol SDK.

## Profiling campaigns
## Performance reports

An installed wheel can populate a self-contained campaign workspace:

```console
pyamplicol profiling-campaign copy ./pyamplicol-profiling-campaign --force
cd ./pyamplicol-profiling-campaign
./steer_performance_campaign.py run \
--workers 1 --table matrix --process-id 1 --multiplicity 1 \
--color-approximation lc --generation-mode non-union-flow \
--generation-engine recurrence --model built_in
```

That deliberately small real campaign measures only the final-state-
multiplicity-one `d d~ > Z` recurrence cell. Broader campaign selections are
intended for dedicated profiling hosts. The
[documentation](https://madgraph.org/pyamplicol/) covers selection,
continuation, optional original-AmpliCol comparisons, artifact retention, and
PDF generation.

The source checkout also contains a thin orchestrator for the dedicated
FullColor FFT comparison. It delegates generation and timing to the existing
pyAmpliCol profiling commands and schedules independent measurement children:

```console
just dev-install --with-legacy-amplicol --with-reference-fft
.venv/bin/python tools/fft_profiling/fft_profiling.py \
--multiplicities 2 3 4 5 \
--lines reference-fft amplicol pyamplicol-recurrence pyamplicol-otf madgraph \
--cores 8 --candidate-cores 1 \
--memory-limit-gib 30 --time-limit-seconds 3600 \
--amplicol-root /path/to/AmpliCol \
--reference-fft-root /path/to/AllGluonsMultipletFFT \
--madgraph-root /path/to/MG5_aMC \
--build-amplicol
```

`--multiplicities` adds the selected values to the persistent fill history and
defaults to `2 ... 9`. `--lines` similarly adds any combination of
`reference-fft`, `amplicol`, `pyamplicol-recurrence`, `pyamplicol-otf`, and
`madgraph`; the two pyAmpliCol groups each schedule their direct and FFT
companion curves together so they reuse the same generation lane. Dependencies
are selected automatically.
Repeating a command against the same output unions both selections, resumes
unfinished cells, and skips completed cells;
`--resume` is an explicit alias for that default. `--cores` is the total
scheduler budget, while `--candidate-cores` is one candidate child's core
claim and evaluator setting. The memory and time limits are strict per-child
cutoffs. `--retry` reruns only failed/skipped cells in the active selection.
`--overwrite` reruns every selected cell and replaces each old result only when
that cell's worker is about to launch; queued or blocked cells retain their old
results. Use `--output PATH` for an independent run directory. `--refresh`
removes only that exact recognized output directory and restarts it, so a
custom output also scopes the refresh; a path that does not exist simply starts
cleanly. Without `--output`, fixed and summed
workloads use separate `IMPLEMENTATION_DOCS/RESULTS/fft-profiling/runs/`
directories named `cluster-fullcolor-n2-n9` and
`cluster-fullcolor-helicity-sum-n2-n9`. Refresh also shares the persistent
MadGraph cache-writer lock and refuses to delete a run while a standalone
MadGraph profiler is using that cache.

Add `--compare-helicity-sums` for the independent complete physical-helicity-
sum workload. The fixed-helicity MadGraph lane selects the shared helicity
through the generated `MATRIX(P,NHEL,IC)` entry point. The summed lane instead
calls the generated `SMATRIX(P,ANS)` with `USERHEL=-1`; MadGraph applies its
native IDEN normalization and may reuse its warmed `GOODHEL` pruning. Fixed-
helicity and summed overlays carry distinct workload identities and cannot be
mixed. `just dev-install` omits the developer-only AmpliCol and Reference FFT
repositories unless they are requested with `--with-legacy-amplicol` and
`--with-reference-fft`; either opt-in also installs the `fft-profiling` Python
extra into `.venv`. Their profiler roots default to
`dependencies/checkouts/legacy-amplicol` and
`dependencies/checkouts/reference-fft`; `--build-amplicol` may build the
AmpliCol probe once. Both paths can be overridden explicitly with
`--amplicol-root` and `--reference-fft-root`. The MadGraph root defaults to
`PYAMPLICOL_MADGRAPH_ROOT` or a recognized developer checkout and may be set
explicitly with `--madgraph-root`.

The published fixed-helicity MadGraph series currently has measured points
through `n=5` for pure gluons and `n=6` for
`d d~ > d d~ + gluons`. Pure-gluon `n=6` retains its measured resource cutoff;
`n=7..9` are explicit protocol-scope not-applicable cells for both families.
The independent helicity-sum MadGraph series has measured points for both
families at `n=2..5`. Every admitted point passed the same-workload numerical
gate before entering the PDFs.

The rolling plot frontiers are per process and implementation, rather than a
claim that every curve reaches the same `n`. Both fixed-helicity and helicity-
sum OTF curves are requested only through final-state `n=6`; beyond that the
publication protocol retains recurrence, AmpliCol, and Reference FFT where
applicable. Within that frontier, cutoffs are annotated rather than hidden or
interpolated. Pure-gluon OTF FFT reached the 3,600 s first-use runtime cap at
`n=6` and retains its measured `n=5` point. The helicity-sum comparison extends
the `d d~ > d d~ + gluons` curves through `n=6` using an authenticated isolated
30 GiB extension. At that point every requested curve measured successfully.
For OTF, direct and FFT setup took 1,725.8 s and 1,607.9 s, while warmed runtime
was 5.321 and 3.759 ms/point respectively (a 1.416x FFT speedup). The measured
peak RSS values were 1.17 and 1.22 GiB.

The plotted *setup time* is deliberately method-specific. pyAmpliCol includes
artifact generation, a fresh load, the first requested evaluation, and OTF
family warm-up where applicable. Reference FFT includes its build,
initialization, and first pass. AmpliCol includes process/color-object
generation for the fixed workload, or process/raw-library generation and build
plus the immutable snapshot for the summed workload. Warmed runtime is measured
separately after those setup boundaries. Resource limits apply individually to
each child. The retained isolated high-frontier extensions use a 30 GiB
process-tree guard.

After the first snapshot is published, rendering never waits for workers and
uses the latest available data. Concurrent renders and refresh publication are
serialized so an older render cannot replace a newer one:

```console
python tools/fft_profiling/fft_profiling.py --render
python tools/fft_profiling/fft_profiling.py --render --compare-helicity-sums
python tools/fft_profiling/fft_profiling.py --render --output /path/to/run
```

For a custom output, the saved manifest determines whether the workload is
fixed-helicity or helicity-summed, so a render does not need to repeat
`--compare-helicity-sums`. Default outputs still use that flag to select the
helicity-sum workspace. Default outputs also refresh the corresponding
canonical PDF; custom outputs keep their PDF inside the selected run directory.
During a scan, the progress display reports the active cell, total live RSS,
and occupied core slots.

An older local MadGraph overlay that predates the node-fingerprint field may
appear only in a nonterminal anytime render, only when its system, machine, and
Python version match the current workstation. Such plots carry an explicit
provenance note; the strict terminal publication merger still requires the
complete host identity produced by a fresh profiler run.

The public performance index retains four selected rendered snapshots. Raw
JSON, generated tables, attempts, and campaign workspaces stay untracked:
Selected manual measurement snapshots are published in the upstream repository:

- [MacBook M3 report](https://github.com/MadGraphTeam/pyamplicol/blob/main/docs/performance_reports/macbook_M3_pyAmpliCol.pdf)
- [AMD EPYC report](https://github.com/MadGraphTeam/pyamplicol/blob/main/docs/performance_reports/EPYC_pyAmpliCol.pdf)
- [FullColor FFT selected-helicity snapshot](https://github.com/MadGraphTeam/pyamplicol/blob/main/docs/performance_reports/summary_plots_final.pdf)
- [FullColor FFT helicity-sum snapshot](https://github.com/MadGraphTeam/pyamplicol/blob/main/docs/performance_reports/summary_plots_final_helicity_sum.pdf)

These are manual measurement snapshots rather than release-CI results; raw
campaign data remains local. The general host report format is reproducible
from an installed package, while the FullColor FFT snapshots use the source-
checkout orchestrator above.

## Documentation

Read the complete [pyAmpliCol documentation](https://madgraph.org/pyamplicol/).
Expand Down
Loading
Loading