Skip to content

Add quantem.atoms: 3D atomic model analysis and atom tracing from volumes - #290

Merged
arthurmccray merged 12 commits into
electronmicroscopy:devfrom
cophus:atom_3d
Oct 26, 2026
Merged

arthurmccray merged 12 commits into
electronmicroscopy:devfrom
cophus:atom_3d

Conversation

@cophus

@cophus cophus commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

What problem this PR addresses

quantem had no tools for analyzing 3D atomic models, the output of atomic electron tomography (AET) or of atomistic simulations, and the atomic tracing on this branch stopped at finding peaks in a volume. This PR adds a quantem.atoms subpackage that takes a set of 3D coordinates through calibration, local structure classification, grain segmentation, strain, and visualization, and completes the tracer in quantem.tomography.atom_trace so that a reconstruction can be converted into such a model. No new required dependencies are added.

Atom tracing (quantem.tomography.Atoms). A volume is modeled as a sum of isotropic 3D Gaussians plus a smooth, non-negative Bezier background, and all site positions, amplitudes, widths and the background are optimized jointly with Adam against the volume. Each Gaussian is splatted only into a window of 3 to 4 of its own sigma around its center, built from three separable 1D factors, so the cost scales with the number of atoms rather than with the number of voxels. Sites are seeded from a difference-of-Gaussians filter and, during refinement, added at residual peaks, removed when weak or isolated, and merged when closer than a fraction of the nearest-neighbor spacing. On a 280^3 volume with 16,000 sites of sigma = 2 voxels, 25 iterations take 4 s on an Apple GPU and 26 s on a laptop CPU.

Atomic model analysis (quantem.atoms).

  • AtomicModel stores the sites in a Vector with named per-site channels, so every measurement (scores, grain labels, strain components, intensities) is a column that the plots and the viewer can select. Coordinates from any file format are passed in as an (N, 3) array; the class has no format-specific loaders.
  • compute_pdf and calibrate measure the radial distribution function with a KD-tree pair count, fit the first peak with an asymmetric generalized Gaussian, and set the physical voxel size from the nearest-neighbor distance of a reference crystal. merge_close_sites removes near-duplicate sites from tracing.
  • templates generates polyhedral templates (fcc, hcp, bcc, sc, diamond/zincblende, wurtzite, icosahedral) from crystal definitions for any number of neighbor shells, together with their proper symmetry rotations, which are found numerically.
  • matching classifies every site by polyhedral template matching, vectorized in torch. Trial rotations are built from pairs of neighbor vectors, each is scored by the one-to-one match between the rotated template and the measured neighbors, and the best rotation is refined with a batched Kabsch solve; an affine fit gives the deformation gradient. Both fcc and hcp templates take about 1 s for 20,000 sites on a laptop CPU. On synthetic fcc, hcp, bcc, diamond and wurtzite particles the classification accuracy is above 97% for interior sites at 3% bond-length noise and about 99% at 5%.
  • segment_grains connects neighboring sites whose disorientation, reduced by the template symmetry with quantem.core.utils.rotations, is below a threshold and labels the connected components, then fills the gaps between grains by majority vote; it also produces a boundary channel (interior, grain boundary, twin, surface). Twin boundaries in fcc appear as hcp-classified sites, since a site on a coherent {111} twin plane has the hcp environment.
  • fit_icosahedral_centers locates the centers of multiply twinned particles as the least-squares intersection of the twin planes carried by the hcp sites, and layer_positions finds the atomic layers along any direction so that plot("slices") can step through a model one layer at a time.
  • compute_strain, surface_distance, sample_volume and classify_species give per-site strain from the affine fit, the distance to the convex hull, the reconstruction intensity around each site, and a species assignment from a one-dimensional Gaussian mixture that leaves low-posterior sites unassigned.
  • structures builds ideal Mackay icosahedra, cuboctahedra, Ino decahedra and mirror-twinned double icosahedra with a chosen center separation, with shell and sector channels, for comparison with measured models and for tests.
  • visualization provides model.plot(kind=...) with the RDF, channel histograms, shaded-sphere slab projections, layer-indexed slice grids, and a template overlay for one site. explode_grains returns a copy with every grain displaced away from its center and the shared boundary sites duplicated into each adjacent grain.
  • model.show() opens an interactive 3D viewer with channel selection, a histogram-driven color range, legend toggles, slab clipping along any normal with a draggable handle, a growth slider, and a site inspector. The viewer itself (ShowAtoms3D) lives in the companion quantem.widget repository; AtomicModel.show() and Atoms.show_3d_atoms() import it only when called, so quantem has no widget dependency.

tests/atoms/test_atoms.py adds 29 tests covering the templates (shell counts and symmetry orders), the RDF fit, the matcher on synthetic lattices (classification accuracy and rotation recovery), the measurements, the ideal structures, the center fit and layer indexing on a synthetic twinned particle, grain explosion, species assignment, the surface classification of open structures, and the end-to-end pipeline on a synthetic fcc twin. tests/tomography/test_atom_trace.py adds 2 tests: the per-site splatting against a dense evaluation, and tracing a synthetic lattice through seeding and refinement with site maintenance. The tracer was also checked on a reconstruction against an independently traced model, recovering 84% of the reference sites with a 0.6-voxel rms offset.

Two small changes outside the new package: Dataset3d.show gains an opt-in same_scale so that all shown slices share one intensity range (2% and 98% quantiles), and quantem/__init__.py imports atoms.

What should the reviewer(s) do

Review the public API of AtomicModel and the module layout, and try the pipeline on a 3D model of your own: from_array, compute_pdf, calibrate, find_neighbors, match_templates(["fcc", "hcp"]), segment_grains, then plot("slab", channel="structure"). Two points I would like feedback on:

  1. Whether quantem.atoms should be a top-level subpackage, as here, or live under quantem.tomography next to the tracer. I put it at the top level because the analysis applies to any 3D atomic model, including simulation output.
  2. The default classification rule. The soft score is the default; an RMSD-based rule and a neighbor-smoothed rule are available, and none of them is reliable above about 8% bond-length noise.

Open item: model.show() needs ShowAtoms3D released in quantem.widget; everything else in quantem works without it.

  • This PR introduces a public-facing change (e.g., figures, CLI input/output, API).
    • For functional and algorithmic changes, tests are written or updated.
    • Documentation (e.g., tutorials, examples, README) has been updated.
    • A tracking issue or plan to update documentation exists: a tutorial notebook on a synthetic twinned particle will follow once the API settles.

This comment was marked as duplicate.

@arthurmccray arthurmccray left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran the tests and tried a few edge cases with Claude. Inline comments below:

Main issues:

  • wurtzite NN factor is off by 1.63x, so calibrate(structure="wurtzite") gives the wrong sampling
  • merge_close_sites(mode="merge") averages categorical channels
  • the surface threshold (< 9 neighbors) only works for close-packed structures
  • explode_grains crashes with an array origin
  • remove_isolated has no fallback when spacing estimation fails
  • the 3D viewer class name is inconsistent between Atoms and AtomicModel
  • Dataset3d.show(same_scale=True) changes default contrast for every caller
  • splat window size is set by the widest site, which can blow up memory

Smaller things:

  • pdf.find_neighbors raises an AxisError on a single-site model.
  • Calling calibrate() after find_neighbors() / surface_distance() leaves bond_mean, bond_std and surface_distance in the old units.
  • _invalidate() clears _matches but leaves the structure and score channels and _template_specs.
  • ruff format --check wants to reformat atom_trace.py.
  • fit_first_peak computes p and then deletes it unused.
  • structures.__all__ is missing attached_icosahedra and growth_steps.

Existing code that could be reused:

  • Consider using quantem.core.utils.rotations (from #305) for measurements.misorientation / rotation_to_quaternion: misorientation_angle_deg and quat_from_matrix cover the same ground.
  • Greedy min-distance suppression is implemented three times (seed_peaks, Atoms.merge_sites, AtomicModel.merge_close_sites); one shared helper would do.

Comment thread src/quantem/atoms/pdf.py Resolved
Comment thread src/quantem/atoms/atomic_model.py Resolved
Comment thread src/quantem/atoms/structures.py Resolved
Comment thread src/quantem/atoms/atomic_model.py Resolved
Comment thread src/quantem/atoms/atomic_model.py Resolved
Comment thread src/quantem/tomography/atom_trace.py Resolved
Comment thread src/quantem/tomography/atom_trace.py Resolved
Comment thread src/quantem/tomography/atom_trace.py Resolved
Comment thread src/quantem/atoms/atomic_model.py Resolved
- pdf: fix the wurtzite NN factor (3/8 sqrt(8/3) a), keep (N, K) neighbor
  arrays for a one-site model, drop an unused variable
- structures: keep the -1 center sentinel in double_icosahedron sectors;
  export attached_icosahedra and growth_steps
- AtomicModel: merge_close_sites takes categorical channels from the
  heaviest site; _invalidate also drops template specs and geometry
  channels; calibrate refreshes bond and surface distances; the surface
  threshold follows the median coordination of the grain sites; explode_grains
  accepts an array origin; sample_volume handles tensor-backed Dataset3d
- matching: accept device/grad tensors; rmsd ignores unmatched entries
- Atoms: resolve the spacing once in refine with fallbacks for min_distance
  and the isolation radius; splat with separable Gaussians grouped by
  per-site window radius; show_3d_atoms goes through AtomicModel.show
- Dataset3d.show: same_scale defaults to False, uses 2-98% quantiles of the
  shown frames and steps aside for any normalization keyword
- remove the show_atoms shim and the empty atom_analysis module
- tests for each fix and a synthetic tracer test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cophus

cophus commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @arthurmccray. Everything from the review is addressed in aa3e785; the inline threads have the details. The smaller items:

  • pdf.find_neighbors on a single site: the KD-tree query now always returns (N, k) arrays, so a one-site model gets the padded (-1, inf) result.
  • Calibrating after find_neighbors() / surface_distance(): calibrate now recomputes bond_mean, bond_std and surface_distance in the new units.
  • _invalidate(): it now also clears _template_specs and _structure_names and drops the channels computed from the geometry (structure, grain, boundary, score_*, rmsd_*, matched_*, strain_*, bond and surface distances). User channels such as intensity, species and sector are kept.
  • ruff format is applied to atom_trace.py.
  • The unused p in fit_first_peak is removed.
  • structures.__all__ now includes attached_icosahedra and growth_steps.
  • The empty tomography/atom_analysis.py is removed.

Two things are not done yet. Using quantem.core.utils.rotations for misorientation needs #305 merged into this branch first, so we will switch once that lands. The greedy suppression in seed_peaks and Atoms.merge_sites can share one helper, which we will do in a follow-up cleanup. AtomicModel.merge_close_sites clusters connected sites rather than suppressing them, so it stays separate.

cophus and others added 2 commits October 24, 2026 15:47
Vector (electronmicroscopy#272) dropped .array and defaults to float32. Read site data with
.numpy() and store the sites in a float64 Vector so positions and channels
keep their precision; one helper builds the sites Vector.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@arthurmccray
arthurmccray marked this pull request as ready for review October 25, 2026 01:08
@arthurmccray

Copy link
Copy Markdown
Collaborator

I approved and merged #305, so you can make the remaining changes and this should be good as well

cophus and others added 2 commits October 25, 2026 18:46
measurements.misorientation now converts the site and symmetry rotations
with quat_from_matrix and calls misorientation_angle_deg from electronmicroscopy#305.  The
angles agree with the matrix-trace version to 3e-6 deg and are more
accurate near zero.  rotation_to_quaternion is removed; quat_from_matrix
covers it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@arthurmccray arthurmccray left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@arthurmccray
arthurmccray merged commit 5b87fc7 into electronmicroscopy:dev Oct 26, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants