Repository navigation
Add quantem.atoms: 3D atomic model analysis and atom tracing from volumes - #290
Conversation
# Conflicts: # widget/scripts/build.mjs # widget/src/quantem/widget/__init__.py
There was a problem hiding this comment.
I ran the tests and tried a few edge cases with Claude. Inline comments below:
Main issues:
- wurtzite NN factor is off by 1.63x, so
calibrate(structure="wurtzite")gives the wrong sampling merge_close_sites(mode="merge")averages categorical channels- the surface threshold (
< 9neighbors) only works for close-packed structures explode_grainscrashes with an array originremove_isolatedhas no fallback when spacing estimation fails- the 3D viewer class name is inconsistent between
AtomsandAtomicModel Dataset3d.show(same_scale=True)changes default contrast for every caller- splat window size is set by the widest site, which can blow up memory
Smaller things:
pdf.find_neighborsraises an AxisError on a single-site model.- Calling
calibrate()afterfind_neighbors()/surface_distance()leavesbond_mean,bond_stdandsurface_distancein the old units. _invalidate()clears_matchesbut leaves thestructureand score channels and_template_specs.ruff format --checkwants to reformatatom_trace.py.fit_first_peakcomputespand then deletes it unused.structures.__all__is missingattached_icosahedraandgrowth_steps.
Existing code that could be reused:
- Consider using
quantem.core.utils.rotations(from #305) formeasurements.misorientation/rotation_to_quaternion:misorientation_angle_degandquat_from_matrixcover the same ground. - Greedy min-distance suppression is implemented three times (
seed_peaks,Atoms.merge_sites,AtomicModel.merge_close_sites); one shared helper would do.
- pdf: fix the wurtzite NN factor (3/8 sqrt(8/3) a), keep (N, K) neighbor arrays for a one-site model, drop an unused variable - structures: keep the -1 center sentinel in double_icosahedron sectors; export attached_icosahedra and growth_steps - AtomicModel: merge_close_sites takes categorical channels from the heaviest site; _invalidate also drops template specs and geometry channels; calibrate refreshes bond and surface distances; the surface threshold follows the median coordination of the grain sites; explode_grains accepts an array origin; sample_volume handles tensor-backed Dataset3d - matching: accept device/grad tensors; rmsd ignores unmatched entries - Atoms: resolve the spacing once in refine with fallbacks for min_distance and the isolation radius; splat with separable Gaussians grouped by per-site window radius; show_3d_atoms goes through AtomicModel.show - Dataset3d.show: same_scale defaults to False, uses 2-98% quantiles of the shown frames and steps aside for any normalization keyword - remove the show_atoms shim and the empty atom_analysis module - tests for each fix and a synthetic tracer test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks @arthurmccray. Everything from the review is addressed in aa3e785; the inline threads have the details. The smaller items:
Two things are not done yet. Using |
Vector (electronmicroscopy#272) dropped .array and defaults to float32. Read site data with .numpy() and store the sites in a float64 Vector so positions and channels keep their precision; one helper builds the sites Vector. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
I approved and merged #305, so you can make the remaining changes and this should be good as well |
measurements.misorientation now converts the site and symmetry rotations with quat_from_matrix and calls misorientation_angle_deg from electronmicroscopy#305. The angles agree with the matrix-trace version to 3e-6 deg and are more accurate near zero. rotation_to_quaternion is removed; quat_from_matrix covers it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What problem this PR addresses
quantem had no tools for analyzing 3D atomic models, the output of atomic electron tomography (AET) or of atomistic simulations, and the atomic tracing on this branch stopped at finding peaks in a volume. This PR adds a
quantem.atomssubpackage that takes a set of 3D coordinates through calibration, local structure classification, grain segmentation, strain, and visualization, and completes the tracer inquantem.tomography.atom_traceso that a reconstruction can be converted into such a model. No new required dependencies are added.Atom tracing (
quantem.tomography.Atoms). A volume is modeled as a sum of isotropic 3D Gaussians plus a smooth, non-negative Bezier background, and all site positions, amplitudes, widths and the background are optimized jointly with Adam against the volume. Each Gaussian is splatted only into a window of 3 to 4 of its own sigma around its center, built from three separable 1D factors, so the cost scales with the number of atoms rather than with the number of voxels. Sites are seeded from a difference-of-Gaussians filter and, during refinement, added at residual peaks, removed when weak or isolated, and merged when closer than a fraction of the nearest-neighbor spacing. On a 280^3 volume with 16,000 sites of sigma = 2 voxels, 25 iterations take 4 s on an Apple GPU and 26 s on a laptop CPU.Atomic model analysis (
quantem.atoms).AtomicModelstores the sites in aVectorwith named per-site channels, so every measurement (scores, grain labels, strain components, intensities) is a column that the plots and the viewer can select. Coordinates from any file format are passed in as an(N, 3)array; the class has no format-specific loaders.compute_pdfandcalibratemeasure the radial distribution function with a KD-tree pair count, fit the first peak with an asymmetric generalized Gaussian, and set the physical voxel size from the nearest-neighbor distance of a reference crystal.merge_close_sitesremoves near-duplicate sites from tracing.templatesgenerates polyhedral templates (fcc, hcp, bcc, sc, diamond/zincblende, wurtzite, icosahedral) from crystal definitions for any number of neighbor shells, together with their proper symmetry rotations, which are found numerically.matchingclassifies every site by polyhedral template matching, vectorized in torch. Trial rotations are built from pairs of neighbor vectors, each is scored by the one-to-one match between the rotated template and the measured neighbors, and the best rotation is refined with a batched Kabsch solve; an affine fit gives the deformation gradient. Both fcc and hcp templates take about 1 s for 20,000 sites on a laptop CPU. On synthetic fcc, hcp, bcc, diamond and wurtzite particles the classification accuracy is above 97% for interior sites at 3% bond-length noise and about 99% at 5%.segment_grainsconnects neighboring sites whose disorientation, reduced by the template symmetry withquantem.core.utils.rotations, is below a threshold and labels the connected components, then fills the gaps between grains by majority vote; it also produces aboundarychannel (interior, grain boundary, twin, surface). Twin boundaries in fcc appear as hcp-classified sites, since a site on a coherent {111} twin plane has the hcp environment.fit_icosahedral_centerslocates the centers of multiply twinned particles as the least-squares intersection of the twin planes carried by the hcp sites, andlayer_positionsfinds the atomic layers along any direction so thatplot("slices")can step through a model one layer at a time.compute_strain,surface_distance,sample_volumeandclassify_speciesgive per-site strain from the affine fit, the distance to the convex hull, the reconstruction intensity around each site, and a species assignment from a one-dimensional Gaussian mixture that leaves low-posterior sites unassigned.structuresbuilds ideal Mackay icosahedra, cuboctahedra, Ino decahedra and mirror-twinned double icosahedra with a chosen center separation, withshellandsectorchannels, for comparison with measured models and for tests.visualizationprovidesmodel.plot(kind=...)with the RDF, channel histograms, shaded-sphere slab projections, layer-indexed slice grids, and a template overlay for one site.explode_grainsreturns a copy with every grain displaced away from its center and the shared boundary sites duplicated into each adjacent grain.model.show()opens an interactive 3D viewer with channel selection, a histogram-driven color range, legend toggles, slab clipping along any normal with a draggable handle, a growth slider, and a site inspector. The viewer itself (ShowAtoms3D) lives in the companion quantem.widget repository;AtomicModel.show()andAtoms.show_3d_atoms()import it only when called, so quantem has no widget dependency.tests/atoms/test_atoms.pyadds 29 tests covering the templates (shell counts and symmetry orders), the RDF fit, the matcher on synthetic lattices (classification accuracy and rotation recovery), the measurements, the ideal structures, the center fit and layer indexing on a synthetic twinned particle, grain explosion, species assignment, the surface classification of open structures, and the end-to-end pipeline on a synthetic fcc twin.tests/tomography/test_atom_trace.pyadds 2 tests: the per-site splatting against a dense evaluation, and tracing a synthetic lattice through seeding and refinement with site maintenance. The tracer was also checked on a reconstruction against an independently traced model, recovering 84% of the reference sites with a 0.6-voxel rms offset.Two small changes outside the new package:
Dataset3d.showgains an opt-insame_scaleso that all shown slices share one intensity range (2% and 98% quantiles), andquantem/__init__.pyimportsatoms.What should the reviewer(s) do
Review the public API of
AtomicModeland the module layout, and try the pipeline on a 3D model of your own:from_array,compute_pdf,calibrate,find_neighbors,match_templates(["fcc", "hcp"]),segment_grains, thenplot("slab", channel="structure"). Two points I would like feedback on:quantem.atomsshould be a top-level subpackage, as here, or live underquantem.tomographynext to the tracer. I put it at the top level because the analysis applies to any 3D atomic model, including simulation output.Open item:
model.show()needsShowAtoms3Dreleased in quantem.widget; everything else in quantem works without it.