[cophus]

What problem does it solve?

This PR adds DiffractionSim, an interactive electron diffraction simulator, packages its browser build as quantem.diffsim for web pages and Dash apps, and removes abTEM from quantem.widget.

DiffractionSim shows a crystal and its diffraction pattern side by side for any orientation, computed live in the browser with Bloch waves: nanobeam disks, CBED and Kikuchi patterns, with thickness, convergence, precession and energy controls. It uses the Crystal and bloch modules of quantEM, which are released in quantem 0.1.10 (electronmicroscopy/quantem#297), so this PR is no longer waiting on quantEM.

The TEM diffraction tab of the Materials Project website has been broken since the site moved to NumPy 2, because no py4DSTEM release contains the NumPy 2 fixes (materialsproject/crystaltoolkit#547, py4dstem/py4DSTEM#750). We plan to replace it with this simulator. The tab is a Crystal Toolkit component in a Dash app, which cannot take on torch, so the replacement has to run without quantEM's Python stack:

• npm run build:web writes a framework-free ES module of the simulator (export default { render }) into packages/quantem.diffsim, a separate PyPI package with no dependencies. A page passes its structure as CIF text, which replaces the structure menu and the CIF button, and the module computes the reflection list, structure factors and Bloch couplings in the browser. quantem.diffsim.settings() converts pymatgen or ASE structures to CIF (a pymatgen structure as its conventional standard cell with its space group, which is the cell the current tab uses), save_html() writes an offline page, and quantem.diffsim.dash serves the module from the app's Flask server and draws every simulator in the layout with one clientside callback. The simulator shows a "Powered by quantEM" link to the quantEM documentation in its lower right corner.
• CIFs loaded in the browser previously had their absorption fixed at 8 % of the elastic coupling. This matches the mean absorption of quantEM's Weickenmeier-Kohl factors (U0''/U0 of 7 to 15 % for the presets), but the WK ratio U_g''/U_g rises with g, from 1.5 to 6 % for Si and from 6 to 35 % for Au between 0 and 3 1/Å. With the fixed 8 %, the median Bloch beam intensities at 300 to 1000 Å were off by 20 to 60 %, and the transmitted intensity of Au [001] at 1000 Å was 0.49 instead of 0.10. js/diffractionsim-web/wk.ts ports quantEM's compute_WK_factor (core loss, phonon absorption and the Debye-Waller factor), so a CIF now gives the same couplings as the presets.

abTEM was used only for the Lobato scattering factors of the ShowCIF potential preview, and it brings in a large dependency stack including dask. The preview now takes these factors from quantem, and its tables agree with abTEM 1.0.10 to 1e-4. Main removed PlanPtycho after this PR was opened, so the PR no longer touches it, and the crystal extra is now empty (kept so that existing install lines work).

What should reviewer(s) do?

• DiffractionSim (src/quantem/widget/diffraction_sim.py, js/diffractionsim/): the merge with main moved it to HtmlExportMixin and quantem.widget.fallback. Its HTML export has no encoding options, since the page holds no image data.
• The browser build (js/diffractionsim-web/): wk.ts, the coupling loop in cif.ts, and the page options in index.ts (cif, structure_name, title, thermal_sigma_A, show_logo).
• packages/quantem.diffsim, tests/diffsim and .github/workflows/diffsim-release.yml: please say whether a second package in this repository is acceptable, or whether it should move to its own repository. The first release needs a PyPI trusted publisher for quantem.diffsim (this repository, diffsim-release.yml, environment pypi), set up by a maintainer before the diffsim-v0.1.0 tag.

Verification

• pytest: 668 passed and 223 skipped, with quantem 0.1.10 from PyPI. Two failures come from the local environment: test_sync_writes_a_manifest_of_the_pinned_ref_and_every_file syncs from a sibling quantem.gpu checkout at another commit, and test_show2d_sibling_static_output_via_nbconvert picks up an lxml without lxml_html_clean.
• tests/diffsim: 3 passed with Dash 3.4, pymatgen and ASE installed.
• vitest: 389 passed. wk.test.ts checks the WK factors against quantem's compute_WK_factor for O, Si, Cu, Au and Ba at 80 to 300 keV (agreement to 1e-8), and checks that a CIF of the silicon preset reproduces the preset couplings to 1e-5 of the largest coupling.
• Browser physics, nanobeam patterns at a zone axis for 100, 300 and 1000 Å (scratch script): structures loaded from CIF now match the presets exactly for Si, GaAs, Al, Au, Fe and SrTiO3, and to 1 to 3 % for Al2O3. Loading a CIF takes 27 ms (Si) to 111 ms (alpha-Mn, 29,100 reflections).
• python -m build packages/quantem.diffsim: a 1.2 MB wheel holding the two modules and the bundle, with no quantem/__init__.py. Installed alone in Python 3.10, it imports no other package and writes a working page.
• Chromium: a Dash 3.4 app standing in for Crystal Toolkit (a structure dropdown, a server callback and quantem.diffsim.dash) draws Si stored as its primitive cell (shown as the conventional cell along [001]), SrTiO3 and GaN, redraws on each selection, follows the light page theme, and shows the corner link to the quantEM documentation, with no console errors. A save_html page of GaN along [11-20] in CBED mode renders at 375 px width without horizontal scrolling, and a DiffractionSim.export_html page renders the Jupyter frontend offline.
• Not checked: the widget inside JupyterLab after the merge (dark theme, Copy), and the MyST website build.

Written by Colin Ophus and Claude (Claude Code).

[github-actions]

• Delete cophus/quantem.widget:diffraction from its remote if you own it, it is not long-lived, and no other PR uses it.
• Remove the dedicated worktree and local diffraction branch after confirming the worktree is clean.
• Fetch electronmicroscopy/quantem.widget and update local main before starting new work.

[bobleesj]

Either we can wait for architecture refactoring or merge now. up to you. probably I can do this during the weekend or so.