Repository navigation
Torch-native PDF pipeline + origin finder improvements - #248
Conversation
|
@ehrhardtkm before I review, it seems like being merged to |
|
Another question I had was from the PR comment was - what's approx speed/improvement given various sizes? Mostly likely it will be either 512x512x192x192 or 256x256x192x192 to be practical, just ballpark numbers. Ref: |
…uch better on experimental data. also user-set r_cut and max_iter
@bobleesj This PR carries just the PDF code from PR #177 plus improvements/fixes since then, which are listed in this PR's description. If |
|
@ehrhardtkm ah-ha divergence! Often inevitable esp for scientific code - maybe before the hackathon, if possible, we shall have |
|
@bobleesj I don’t think nanobeam can go into dev as a plain merge. Apart from the PDF code it carries 5k lines (MAPED, strain mapping, a file reader, imaging_utils changes). Also, from what I see, its imaging_utils fixes the same sub-pixel cross-correlation bug as dev’s #281, plus open PRs #284 and #97 are also on newer version of the strain code. It would be cleaner for at least the strain and PDF code to go into dev in these separate merges, I think. |
happy to merge this PR to @arthurmccray could you please check if I have permission for merging? 
|
|
@bobleesj I just made you a maintainer, so you should have merge abilities now. Lemme know if it doesn't work for some reason |
|
@arthurmccray thank you. @ehrhardtkm ah, we merge conflicts! |
|
Adding some more updates since the initial PR description, for posterity: Low-r correction now takes the density as an input. Below the nearest-neighbour distance there are no atom pairs, so G(r) should follow the straight line −4πρ₀r there and errors in S(k) show up as spurious oscillations instead. PairDistributionFunction caches the radial mean I(k) and the background fit. Calling Because all scan positions are resampled onto one polar grid, the k range ends where the pattern closest to the detector edge runs out. One wrongly found center could therefore shorten it for the whole dataset. It now uses the 0.1th percentile of the positions instead of the worst one, and warns if positions lose data. Elliptical distortion fit (new). Also, some timing benchamrking that @bobleesj asked for, for the full PDF pipeline (origin finding → polar transform → PDF) Before is PR#177:
Will work on the merge conflicts shortly |
|
@bobleesj Let me know if you have any other comments or concerns! |
Could you please check my status? Seems like my prev review rights were revoked. |
|
can you not do a review and approve it? I made you a maintainer so you should be able to? |
oh thanks. Sorry Arthur - didn't know we had to submit review for PR approval. All good. |
bobleesj
left a comment
bobleesj
left a comment
There was a problem hiding this comment.
Merging since it has no side effects on other sets of algos. Thank you!
* feat: add ShowPDF for pair distribution function analysis Add ShowPDF, an interactive front end to quantem's PairDistributionFunction (electronmicroscopy/quantem#248) for ePDF analysis of 4D-STEM scans. - Select scan positions with a painted mask, a square probe, a line band, or a line scan with one curve per position, and see I(k) with its background, F(k), G(r) or g(r) of the selection. - The settings are passed to calculate_Gr (fit range, k window, filters, r grid, low-r correction). The number density is an input, never estimated. - Line widths are odd, so a horizontal or vertical band of width w covers exactly w rows or columns. A line scan has a position at every scan pixel along the line's longer axis. - Standalone HTML export follows the html-export protocol: "full" keeps the k-space and correction controls live; "uint8" also embeds every position's radial profile, so the selection stays live too. - Saved notebooks follow the save-state contract: by default they keep a PNG preview instead of the scan image, mask and line-scan stack; save_state=True keeps everything. Exports always embed them. - The Python tests skip until quantem provides the #248 API; the tests on exported HTML run in the browser tier (QUANTEM_WIDGET_BROWSER=1). - docs/tutorials/showpdf.ipynb and datasets.showpdf_ta are included but not in the docs sidebar yet: the Ta scan waits for a quantem-data pull request. * chore: pin the ShowPDF tutorial data to its quantem-data commit The simulated amorphous Ta scan is now on bobleesj/quantem-data (4dstem/ta_amorphous_sim, added by its pull request 5 in commit fe907499). showpdf_ta and showpdf_ta_ground_truth read that commit by default, and the docs and changelog no longer say the data is unpublished.
What problem this PR addresses
Follow-on to #177 (the polar transform + PDF pipeline that merged into
nanobeam). This PR brings that work ontodevand lands the changes I've made since.Conceptually unchanged — same
Polar4dstemdatastructure, samePairDistributionFunctionshape, same overall workflow (origin → polar → PDF). The public API should be the same, except for a few minor changes.Changes since #177:
Fully torch-native, dual-backend pipeline. The whole pipeline accepts numpy or torch input on the existing
Dataset, device is inferred from the input, and there are no GPU↔CPU round-trips inside the pipeline. numpy vs tensor input give identical results.Background fitting. Rewritten with an analytic Jacobian that is noticeably faster than the previous implementation. Added a batched
fit_bg_batchedpath for full-dataset use, which the downstream PDF widget (PR3) needs to recompute backgrounds interactively. Also includes some small fixes I picked up while running this on real data.Origin finding. The existing finder's hardcoded margin requirement caused it to error out on smaller detectors, so I made it the margin dynamic and added a 2D-quadratic subpixel refinement (renamed to
find_origin_angular_grid). I also added a secondary option,find_origin_angular_descent, a margin-free, COM-anchored descent finder. It's slower but I've found it slightly more reliable on noisier data. Both are dispatched fromPolar4dstem.from_data(origin_method="grid"|"descent").Tests in
tests/diffraction/test_polar.pyare parametrized over both origin finders and add coverage for the new code paths.Out of scope (follow-on PRs to shortly follow):
What should the reviewer(s) do
Merge after review. Reviewing commit-by-commit is likely easier than the unified diff — each commit is a single conceptual change.