Skip to content

feat: add ShowPDF for pair distribution function analysis - #246

Merged
bobleesj merged 2 commits into
electronmicroscopy:mainfrom
ehrhardtkm:oct-18-add-showpdf
Oct 19, 2026
Merged

bobleesj merged 2 commits into
electronmicroscopy:mainfrom
ehrhardtkm:oct-18-add-showpdf

Conversation

@ehrhardtkm

@ehrhardtkm ehrhardtkm commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

What problem does it solve?

Electron PDF analysis of amorphous and nanocrystalline 4D-STEM data needs the
reduction (background fit, k window, filters, low-r correction) tuned for a region of
the scan, and the result checked as G(r) and g(r). quantem's
PairDistributionFunction (electronmicroscopy/quantem#248) does the reduction;
ShowPDF makes it interactive. You select scan positions on a bright-field image and
the curves of that selection follow as you change the settings. Every curve in a
notebook is what the library computes.

electronmicroscopy/quantem#248 is on quantem's dev branch until the next quantem
release. Until then the ShowPDF tests skip in CI (quantem 0.1.10), and constructing a
ShowPDF raises an ImportError that names the requirement.

What should reviewer(s) do?

  • Read docs/api/showpdf.md: the behavior, how the traits map onto calculate_Gr,
    and the errors.

  • In src/quantem/widget/showpdf.py, _curve_params builds the calculate_Gr
    arguments in one place; _effective_line, _linescan_samples and
    _compute_linescan define the line scan; _offline_profiles builds the
    per-position profiles that the "uint8" export embeds.

  • js/showpdf/pdf_compute.ts repeats the library's steps for exported pages, and
    js/showpdf/linescan.ts mirrors the Python line geometry.
    pdf_compute.numpy.test.ts and linescan.numpy.test.ts compare them with the
    Python results.

  • CommitSlider in js/showpdf/index.tsx works around MUI committing a stale value
    after a click that changed nothing; its comment states its rules.

  • Saved notebooks: ShowPDF mixes in SavedStateMixin and StaticFallbackMixin
    (_UNSAVED_HEAVY_KEYS, _static_png_b64, a PNG preview like Plot2D's), and
    ShowPDFRoot shows that preview when a reopened model has no scan image. Its
    export embeds self, so _write_html_export turns the trim off for that snapshot.

  • The tutorial is in this PR but held back. docs/tutorials/showpdf.ipynb and the
    loaders datasets.showpdf_ta and datasets.showpdf_ta_ground_truth are included,
    but the notebook is not in _toc.yml or linked yet, so the docs build skips it. Its
    data, a simulated amorphous Ta scan with the g(r) of its atomic model, is on
    bobleesj/quantem-data (4dstem/ta_amorphous_sim, added by
    dataset pull request #5),
    and the loaders read the commit that added it. Two things outside this PR hold it
    back:

    1. Main's tutorial loaders still read widget-tutorials/, which the dataset no
      longer has since its reorganization on 2026-10-11, so they raise
      FileNotFoundError today. The last docs build on main ran before that, and the
      next one will fail on those tutorials, with or without this PR. The ShowPDF
      loader already reads the current <modality>/<name> layout.
    2. The docs build installs quantem from PyPI, which lacks
      Torch-native PDF pipeline + origin finder improvements quantem#248 until the next release.

    Once these are resolved, a follow-up adds the notebook to _toc.yml and links it
    from the README, the docs intro and the API index.

  • Not ShowPDF, a heads-up: quantem 0.1.10, on PyPI since 2026-10-16 (after main's last
    CI run), replaces r_to_q_rotation_cw_deg in Dataset4dstem metadata with
    q_to_r_rotation_ccw_deg and adds q_transpose. So
    test_platform.py::test_stand_in_carries_the_quantem_core_surface_the_widgets_use
    fails in CI now, on main as well as on this branch.

Verification

macOS arm64, quantem dev at 6d9474c (the merge of electronmicroscopy/quantem#248),
quantem.gpu at gpu-ref.txt's ref:

  • PYTHONPATH=src pytest -q: 729 passed, 112 skipped, 4 failed. The 4 failures fail
    the same way on clean main: the stand-in metadata test above,
    test_phase_calibration_is_verified_by_ring_ratios and two ShowPtycho tests.
  • pytest tests/showpdf tests/test_html_export_protocol.py tests/test_tutorial_data.py tests/test_save_state.py:
    359 passed, including the seven save-state contract checks for ShowPDF.
  • CI's setup (Python 3.12, -e ".[test]" with quantem 0.1.10 from PyPI,
    QUANTEM_GPU_SRC at gpu-ref.txt's ref): PYTHONPATH=src pytest -q gives 438 passed,
    90 skipped, 2 failed. The ShowPDF test modules skip, since 0.1.10 lacks
    calculate_Gr(density=..., max_iter=...), and the Ta loader tests pass. Both
    failures fail the same way on clean main in that setup: the stand-in metadata test
    above and test_phase_calibration_is_verified_by_ring_ratios (which passed in main's
    last CI run, on Linux). Both size guards pass.
  • npm run build and npm run typecheck pass. The 50 ShowPDF tests of npm test pass,
    with their NumPy comparisons. On this machine under load, some of main's timing tests
    (js/show2d/colormap.test.ts, js/display/levelEdges.test.ts) time out at vitest's
    5 s; they involve no ShowPDF code.
  • QUANTEM_WIDGET_BROWSER=1 pytest tests/browser/test_showpdf_browser.py: 47 passed.
    These drive exported HTML in headless Chromium and need the network for the CDN
    scripts; on a busy machine a page load can time out at 60 s, and a rerun passes.
  • Live JupyterLab 4.5 on the Ta scan: mask, probe and line selection, the line scan as
    heatmap and waterfall, the plot modes and a slider commit work in the light and dark
    themes. After Cmd+S (with widget state saved), a kernel shutdown and a reopen, the widget
    restores without rerunning; without saved widget state the notebook shows the static
    preview.
  • Simulated amorphous Ta (20 × 20 scan, k to 2.26 1/Å): ShowPDF's g(r) equals the
    library's exactly (max |Δg| = 0) with the tutorial's settings. The tutorial runs end
    to end with the download pointed at the local files, whose hashes match the files
    at the pinned dataset commit; its g(r) has the model's peak positions, with lower
    and broader peaks from the k range.

This PR follows the
scikit-package workflow for
reproducible scientific software.

Add ShowPDF, an interactive front end to quantem's PairDistributionFunction
(electronmicroscopy/quantem#248) for ePDF analysis of 4D-STEM scans.

- Select scan positions with a painted mask, a square probe, a line band, or a
  line scan with one curve per position, and see I(k) with its background,
  F(k), G(r) or g(r) of the selection.
- The settings are passed to calculate_Gr (fit range, k window, filters, r grid,
  low-r correction). The number density is an input, never estimated.
- Line widths are odd, so a horizontal or vertical band of width w covers
  exactly w rows or columns. A line scan has a position at every scan pixel
  along the line's longer axis.
- Standalone HTML export follows the html-export protocol: "full" keeps the
  k-space and correction controls live; "uint8" also embeds every position's
  radial profile, so the selection stays live too.
- Saved notebooks follow the save-state contract: by default they keep a PNG
  preview instead of the scan image, mask and line-scan stack; save_state=True
  keeps everything. Exports always embed them.
- The Python tests skip until quantem provides the electronmicroscopy#248 API; the tests on
  exported HTML run in the browser tier (QUANTEM_WIDGET_BROWSER=1).
- docs/tutorials/showpdf.ipynb and datasets.showpdf_ta are included but not in
  the docs sidebar yet: the Ta scan waits for a quantem-data pull request.
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
  • Delete ehrhardtkm/quantem.widget:oct-18-add-showpdf from its remote if you own it, it is not long-lived, and no other PR uses it.
  • Remove the dedicated worktree and local oct-18-add-showpdf branch after confirming the worktree is clean.
  • Fetch electronmicroscopy/quantem.widget and update local main before starting new work.

The simulated amorphous Ta scan is now on bobleesj/quantem-data
(4dstem/ta_amorphous_sim, added by its pull request 5 in commit fe907499).
showpdf_ta and showpdf_ta_ground_truth read that commit by default, and the
docs and changelog no longer say the data is unpublished.

@bobleesj bobleesj left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merged @ehrhardtkm thanks!

@bobleesj
bobleesj merged commit 11273ad into electronmicroscopy:main Oct 19, 2026
2 of 3 checks passed
@ehrhardtkm
ehrhardtkm deleted the oct-18-add-showpdf branch October 19, 2026 20:00
bobleesj added a commit that referenced this pull request Oct 21, 2026
…red display code

Everything since rc41 in one commit (343 commits on the fork). The changelog's
Unreleased section lists the changes, including the rc42 and rc43 work that was
prepared but never published to TestPyPI.

SSB (with quantem.gpu)
- Show4DSTEM Compute SSB and ShowPtycho fit model-based and tilt-corrected by
  default. The other two methods are explicit options under quantem.gpu's names:
  tilt=False, and objective="phase_variance" as a backup.
- SSB phases are shown at the automatic output factor; fits stay on the native
  grid. Exports say when the automatic factor is rounded up for the browser.
- ShowPtycho: save_calibration() from Python, saved data named portably and
  restored by content, saves reopened after the 180-degree branch move, Crop and
  Refit through quantem.gpu's select_region, and a trial strip listing every
  trial with its objective.
- Show4DSTEM presets sit on the recorded bright-field disk. Flagged detector
  pixels are filled by the neighbour median, as quantem.gpu does, and announced.
- Vocabulary follows quantem.gpu's names through one adapter. A one-time script
  rewrites earlier SSB calibration files.

Interactivity
- ShowPtycho: two slider requests in flight, one display FFT at a time prepared
  in its worker, no loss while the rotation slider drags, one repaint per
  animation frame, and unchanged canvas sets skipped.
- Show4DSTEM: one render per virtual image and per animation frame during a
  detector drag.
- An interactivity benchmark drives both widgets in Chrome against a per-chip
  baseline (Apple M5 Max recorded).

Shared display code
- One export save dialog and payload writer across five widgets, shared toolbar
  styles, touch helpers, page navigation, FFT helpers, colormap shaders and
  overlay validators. About 1.5k fewer lines.
- Show3D filters a frame's panels concurrently, and the WebGPU frequency filter
  reuses its mask pipeline. Many Show1D, Show2D, Show3D, Show3DSlices and
  ShowDiffraction fixes.

Docs and tutorials
- The SSB workflow, ShowPtycho and Show4DSTEM tutorials follow the model-based
  default, with detector calibration as its own step and the rotation branch
  settled first.

ShowPDF (#246) is kept as merged upstream. Four lines adapt it to the renamed
static-fallback trait (_static_fallback_image) and envelope field (schema), and
shared export/state helpers it uses (ensure_mobile_viewport, request mode,
resolve_widget_version, unwrap_state_payload) are restored. The version is
0.0.1rc43, the version this work was prepared as (quantem.live requires
quantem.widget>=0.0.1rc43); rc42 and rc43 were never published to TestPyPI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants