Add DiffractionSim and quantem.diffsim, and remove the abTEM dependency

.github/workflows/diffsim-release.yml

@@ -0,0 +1,66 @@
+name: quantem.diffsim release
+
+# quantem.diffsim (packages/quantem.diffsim) is the browser build of DiffractionSim for web pages and
+# Dash apps. It has no dependencies, so it is published to PyPI on its own, from a diffsim-vX.Y.Z tag.
+# PyPI needs a trusted publisher for this repository, workflow and the "pypi" environment.
+
+on:
+  push:
+    tags:
+      - "diffsim-v*"
+
+permissions:
+  contents: read
+  id-token: write
+
+concurrency:
+  group: diffsim-release-${{ github.ref }}
+  cancel-in-progress: false
+
+jobs:
+  pypi:
+    name: Build and publish quantem.diffsim
+    runs-on: ubuntu-latest
+    environment: pypi
+
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v7
+
+      - uses: actions/setup-python@v6
+        with:
+          python-version: "3.12"
+
+      - uses: actions/setup-node@v6
+        with:
+          node-version: "22.x"
+          cache: npm
+          cache-dependency-path: package-lock.json
+
+      - name: Check the tag against the package version
+        run: |
+          version=$(python -c "import tomllib; print(tomllib.load(open('packages/quantem.diffsim/pyproject.toml', 'rb'))['project']['version'])")
+          test "diffsim-v${version}" = "${GITHUB_REF_NAME}" || { echo "tag ${GITHUB_REF_NAME} does not match version ${version}"; exit 1; }
+
+      # the web-page bundle only, which needs no quantem.gpu sources (the widget bundles do)
+      - name: Build the browser bundle
+        run: |
+          npm ci
+          node scripts/build.mjs --web-only
+          test -s packages/quantem.diffsim/src/quantem/diffsim/static/diffraction-sim.js
+
+      - name: Build distributions
+        run: |
+          python -m pip install --upgrade build twine
+          python -m build packages/quantem.diffsim --outdir dist/diffsim
+          python -m twine check dist/diffsim/*
+
+      - name: Check the wheel installs and serves the bundle
+        run: |
+          python -m pip install dist/diffsim/quantem_diffsim-*.whl ase
+          python -c "import quantem.diffsim as d; from ase.build import bulk; assert d.bundle_path().stat().st_size > 500_000; assert 'quantem-diffsim-bundle' in d.html(bulk('Si', cubic=True))"
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          packages-dir: dist/diffsim/

.github/workflows/widget-ci.yml

@@ -78,6 +78,7 @@ jobs:
         run: |
           npm ci
           npm run build
+          npm run build:web  # the quantem.diffsim bundle, for tests/diffsim
 
       - name: Typecheck and unit-test the frontend
         run: |

.github/workflows/widget-release.yml

@@ -124,6 +124,7 @@ jobs:
           wheel = wheels[0]
           required = {
               "quantem/widget/static/chooselattice.js",
+              "quantem/widget/static/diffractionsim.js",
               "quantem/widget/static/engine-manifest.json",
               "quantem/widget/static/show1d.js",
               "quantem/widget/static/plot2d.js",

.gitignore

@@ -16,6 +16,7 @@ js/.generated/
 src/quantem/widget/static/*.js
 src/quantem/widget/static/*.css
 src/quantem/widget/static/engine-manifest.json
+packages/quantem.diffsim/src/quantem/diffsim/static/*.js
 
 # Docs build
 docs/_build/

CHANGELOG.md

@@ -6,6 +6,9 @@ new `rcN` heading when that rc is published to TestPyPI.
 
 ## Unreleased
 
+- Add `DiffractionSim`: rotate a unit cell and see its nanobeam, CBED or Kikuchi pattern update live, with kinematical or Bloch wave intensities computed in the browser, state save/load, and HTML export. It needs quantem 0.1.10 or later (`quantem.diffraction.Crystal` and `bloch`).
+- Add `quantem.diffsim`, a separate PyPI package with no dependencies, built from `packages/quantem.diffsim`: the browser build of DiffractionSim (`npm run build:web`) for web pages and Dash apps, with `settings()` and `save_html()` for pymatgen, ASE or CIF structures and `quantem.diffsim.dash` for Dash. A page can pass its structure as CIF text, which replaces the structure menu, and shows a "Powered by quantEM" link to the quantEM documentation. CIFs loaded in the browser now get the absorptive Weickenmeier-Kohl couplings with a Debye-Waller factor, ported from quantem, instead of an absorption fixed at 8 % of the elastic coupling, which was up to 5 times too strong for the low-order reflections of light elements and up to 4 times too weak for the high-order reflections of gold.
+- Drop abTEM. The ShowCIF potential preview takes its Lobato scattering factors from quantem (the tables agree with abTEM's to 1e-4), so the `crystal` extra is now empty and kept only so existing install lines work.
 - ShowPtycho's trial strip lists the default fit's trials (model agreement, each with its own sample tilt and depth spread) as well as phase-variance ones; it showed only phase-variance trials, so it was empty for every default fit. Each tile names its objective and score (agreement higher is better, phase variance lower), trials rank within one objective, and clicking a trial applies its optics and sample model.
 - Show4DSTEM Compute SSB runs quantem.gpu's default fit, the SSB model with the sample tilt and depth spread (tilt-corrected); it ran the tilt-0 fit with Nelder-Mead. `compute_ssb(tilt=False)` fits the same model at tilt 0 and `compute_ssb(tilt=False, objective="phase_variance")` is the older phase-variance SSB; the new **Fit** menu (`ssb_compute_tilt`, `ssb_compute_objective`) picks one of the three, and the record names the `objective`. `refinement` takes quantem.gpu's `"auto"` (default), `"newton"`, `"nelder-mead"` or None, and the record names the optimizer that ran. Locks in the tilt-corrected fit are quantem.gpu's `fixed_aberrations` (C10 alone, C12 at 0 alone, or both locks for all three). A manual recompute keeps the fitted tilt and depth spread. Record `quantem.show4dstem-ssb-calibration/3` adds `tilt`, `sample_tilt_mrad`, `depth_spread_nm` and `tilt_fit_gain`; `scripts/rewrite_ssb_calibration.py` carries /2 files over as tilt 0.
 - Show4DSTEM places its BF, ABF, ADF and HAADF presets on the bright-field disk `data.calibration.fit_detector_sampling()` recorded, else on the same fit (`quantem.gpu.detector.fit_bright_field_disk`): the radius is now the disk edge, where the convergence semiangle sits, instead of the equal-area circle above mean + std (the presets of a blurred disk move by a few percent). A disk given as `bright_field_center_px` / `bright_field_radius_px` still wins.

README.md

@@ -121,8 +121,7 @@ tensors on any device.
 Follow the [crystal-to-volume demo route](docs/tutorials/demo.md): inspect a
 crystal with **ShowCIF**, then explore depth using **Show3D** and **Show3DSlices**.
 The notebooks include runnable models, physical units, and controls to try.
-The ShowCIF potential preview needs the `crystal` extra (abTEM); new APIs
-require a source build containing them. The guide includes setup and browser requirements.
+New APIs require a source build containing them. The guide includes setup and browser requirements.
 
 ## Widgets
 
@@ -139,6 +138,7 @@ require a source build containing them. The guide includes setup and browser req
 | `ShowPDF` | Pair distribution function analysis of 4D-STEM data by scan region | [API](https://electronmicroscopy.github.io/quantem.widget/api/showpdf.html) |
 | `ChooseLattice` | Select an origin and lattice vectors | [API](https://electronmicroscopy.github.io/quantem.widget/api/choose-lattice.html) |
 | `ShowCIF` | Crystal repeats, columns, tilt, and projected potential/phase previews | [tutorial](docs/tutorials/showcif.ipynb) · [API](docs/api/showcif.md) |
+| `DiffractionSim` | Rotate a crystal and see its nanobeam, CBED, or Kikuchi pattern, computed in the browser | [API](docs/api/diffractionsim.md) |
 
 ## Documentation
 

docs/_toc.yml

@@ -82,6 +82,7 @@ parts:
           - file: api/showpdf
           - file: api/choose-lattice
           - file: api/showcif
+          - file: api/diffractionsim
           - file: api/html-export
           - file: api/io
           - file: api/load

docs/api/diffractionsim.md

@@ -0,0 +1,186 @@
+# DiffractionSim
+
+Rotate a unit cell and watch its electron diffraction pattern follow. The left
+panel draws the cell, the right panel the nanobeam pattern, CBED disks, or the
+Kikuchi pattern of the same orientation. Kinematical and Bloch wave
+intensities are computed in the browser, so dragging the crystal needs no
+Python round trip.
+
+```python
+from quantem.widget import DiffractionSim
+
+sim = DiffractionSim("Si (diamond cubic)", zone_axis=[1, 1, 0])
+sim
+```
+
+The crystal can be a preset name (`DiffractionSim.presets_available()`), a CIF
+path, an ASE `Atoms` object, or a quantem `Crystal`.
+
+## Requirements
+
+DiffractionSim needs quantem 0.1.10 or later, for `quantem.diffraction.Crystal`
+and `quantem.diffraction.bloch`. quantem is a base dependency of
+quantem.widget except on Intel Macs and Windows on ARM; there, constructing the
+widget raises an `ImportError` that names this requirement, and the
+DiffractionSim tests are skipped.
+
+This widget is unrelated to [ShowDiffraction](showdiffraction), which measures
+experimental patterns.
+
+## Reference
+
+```{eval-rst}
+.. autoclass:: quantem.widget.diffraction_sim.DiffractionSim
+   :members: presets_available, crystal, set_crystal, set_zone_axis, compute_kossel_reference, state_dict, load_state_dict, save, export_html
+```
+
+## Physics
+
+`energy_keV` and `k_max_inv_A` set the reflection list, computed in Python:
+the reciprocal lattice points within `k_max_inv_A`, each with its kinematical
+`|F_g|^2` (Lobato and Van Dyck factors) and its Bloch coupling `U_g`
+(absorptive Weickenmeier and Kohl factors at `energy_keV`). Changing either
+trait recomputes the list.
+
+With `dynamical=True`, reflections within `sg_max_inv_A` of the Ewald sphere
+enter a Bloch wave calculation; the rest take thin-slab intensities. With
+`dynamical=False`, every reflection takes `|F_g|^2` with a Gaussian excitation
+envelope of width `sigma_excitation_inv_A`. `precession_deg` averages the
+pattern over `n_precession` incident tilts on the precession cone.
+
+`mode` is `"nanobeam"`, `"cbed"` or `"kikuchi"`. `"kossel"` is accepted as the
+earlier name of `"kikuchi"` and stored as `"kikuchi"`. The pixel rendering of
+the Kikuchi mode needs a reference pattern from `compute_kossel_reference()`
+(about a minute for silicon at 3 mrad).
+
+Units are in the trait names: `_keV`, `_A` (Å), `_inv_A` (1/Å), `_mrad`,
+`_deg`, `_px` (CSS pixels), `_pct` (percent).
+
+### Data sources
+
+| Data | Source | Used for |
+|---|---|---|
+| Kinematical scattering factors | Lobato & Van Dyck, Acta Cryst. A 70, 636 (2014), [doi:10.1107/S205327331401643X](https://doi.org/10.1107/S205327331401643X), from quantem | `\|F_g\|^2` in Python; `js/diffractionsim-web/lobato.ts` holds the same parameters, copied from quantem's `data/lobato.json` (MIT license), for CIFs loaded in the web build |
+| Absorptive Bloch couplings | Weickenmeier & Kohl, Acta Cryst. A 47, 590 (1991), [doi:10.1107/S0108767391004774](https://doi.org/10.1107/S0108767391004774), from quantem | `U_g` of every crystal set up in Python, including the web build presets; `js/diffractionsim-web/wk.ts` ports quantem's `compute_WK_factor` (MIT license) for CIFs loaded in the web build |
+| Atom colors and radii | ASE (`jmol_colors`, `covalent_radii`) | Cell drawing |
+
+CIFs loaded in the browser (web build only) get the same absorptive
+Weickenmeier-Kohl couplings, with core loss, phonon absorption and a
+Debye-Waller factor for an RMS thermal displacement of 0.05 Å
+(`thermal_sigma_A`). The factors are tabulated every 0.0025 1/Å and
+interpolated; a CIF of the silicon preset reproduces the preset couplings to
+1e-5 of the largest coupling (`js/diffractionsim-web/wk.test.ts`).
+
+## Interactive controls
+
+| Control | Trait | Expected effect |
+|---|---|---|
+| Crystal menu | `preset` | Loads the preset and recomputes the reflection list |
+| Energy menu | `energy_keV` | Recomputes the reflection list at the new energy |
+| Zone axis field, Go | `orientation` | Puts [uvw] (or [uvtw] for hexagonal cells) along the beam |
+| Copy | none | Copies both panels as one PNG |
+| Export | none | PNG of both panels, or a standalone HTML page |
+| Reset | `orientation` | Returns to the identity orientation |
+| Drag the cell | `orientation` | Tilts the crystal; the near face follows the pointer |
+| Drag the pattern | `orientation` | Tilts the crystal so the pattern follows the pointer |
+| Shift-drag or two-finger twist | `orientation` | Rotates about the beam |
+| Double-click a disk | `orientation` | Tilts to the two-beam condition of that reflection |
+| Double-click empty space | `orientation` | Moves the Laue circle centre to the click |
+| x, y, z buttons and Step | `rotation_step_deg` | Rotates about the screen axes by the step |
+| Spin ↔ ↕ and speed | `rotation_speed_deg_per_s` | Rotates continuously about a screen axis |
+| Cell Axes, Axis Labels, Polyhedra, Ewald Sphere | `show_cell_axes`, `show_labels`, `polyhedra`, `show_ewald` | Cell panel overlays |
+| Cells | `n_cells` | Number of cells drawn along a, b, c (1 to 6) |
+| Nanobeam / CBED / Kikuchi | `mode` | Pattern type |
+| Markers / Disks / Pixels (Lines / Pixels) | `render` | Pattern rendering |
+| Dynamical | `dynamical` | Bloch wave or kinematical intensities |
+| hkl Labels | `show_hkl` | Labels the strongest reflections or lines |
+| Kikuchi Lines | `show_kikuchi` | Overlays Kikuchi line pairs on nanobeam and CBED |
+| Quality | `quality` | Bloch beam cap and CBED tilt sampling |
+| Compute Reference | `kossel_json` | Computes the Kikuchi reference pattern in Python |
+| Thickness | `thickness_A` | Specimen thickness |
+| Semiangle | `semiangle_mrad` | CBED and nanobeam disk radius |
+| Precession | `precession_deg` | Precession half angle |
+| Pattern Range | `pattern_range_inv_A` | Scattering vector at the panel edge |
+| Field Half-Angle | `field_mrad` | Kikuchi pattern half angle |
+| Excitation Error σ | `sigma_excitation_inv_A` | Kinematical envelope width (Dynamical off) |
+| Marker sliders | `marker_power`, `marker_size_px` | Marker area exponent and largest radius |
+| Scale, Exponent, Color, histogram | `scaling`, `power`, `cmap`, `vmin_pct`, `vmax_pct` | Pixel rendering display |
+
+`cmap` takes the shared widget colormaps, their `_r` reversals, and
+`turbo_black` and `turbo_black_r`, whose lookup tables come from
+`quantem.core.visualization` (trait `cmap_luts`).
+
+## State
+
+```python
+sim.save("si_110.json")
+sim2 = DiffractionSim(state="si_110.json")
+```
+
+`state_dict()` holds the settings that restore the view: preset, energy,
+orientation, mode and every display setting. It does not hold the reflection
+list or the Kikuchi reference pattern, which are recomputed. A crystal from a
+CIF or ASE is passed again: `DiffractionSim("my.cif", state="my.json")`.
+
+With `save_state=False` (the default), a saved notebook keeps the settings and
+a PNG of both panels, which the frontend captures after the view settles, and
+leaves out the Kikuchi reference pattern. `save_state=True` embeds everything.
+
+## HTML export
+
+```python
+sim.export_html("si_110.html", presets=["Al (fcc)", "Ti (hcp)"])
+```
+
+The page follows the [HTML export](html-export) protocol and carries the widget
+runtime, so it opens with no network. All simulations run in the browser:
+rotation, thickness, mode and display changes need no Python. The beam energy
+and `k_max_inv_A` are fixed to the exported values; `presets` embeds
+additional structures for the crystal menu. The Export menu in the widget
+writes the same page through the export bridge. The page holds no image data,
+so there are no encoding or binning options.
+
+## Web pages and Dash apps
+
+`npm run build:web` (not `npm run build`) writes
+`packages/quantem.diffsim/src/quantem/diffsim/static/diffraction-sim.js`, a
+single ES module with `export default { render }`, built from
+`js/diffractionsim-web/` without React. Every calculation runs in the browser,
+including the structure factors of a structure given as CIF text, so a page
+needs no Python and no quantem. The module is published on PyPI as
+`quantem.diffsim`, which has no dependencies and adds a few helpers:
+
+```python
+import quantem.diffsim as diffsim
+
+diffsim.save_html("si.html", structure, zone_axis=[1, 1, 0], title="Si")  # one offline page
+diffsim.settings(structure, zone_axis=[0, 0, 1], mode="nanobeam")         # the dict the module reads
+```
+
+`structure` can be a pymatgen `Structure` (written as its conventional
+standard cell, with its space group), an ASE `Atoms` object, CIF text or a
+CIF path. A structure passed this way replaces the structure menu and the
+CIF button. The page also shows a "Powered by quantEM" link to the quantEM
+documentation (`show_logo=False` removes it).
+
+In a Dash app, `quantem.diffsim.dash.register(app)` serves the module from
+the app's Flask server and adds one clientside callback, and
+`quantem.diffsim.dash.layout(name)` places a simulator. A server callback that
+writes `diffsim.settings(...)` to `quantem.diffsim.dash.settings_id(name)`
+redraws it, for example when the page shows a new structure. The package
+README has a complete example.
+
+The preset structure menu is `js/diffractionsim-web/presets.ts` (about 1 MB),
+generated by `scripts/diffraction_sim_presets.py` at 200 keV and `k_max` 3.0
+1/Å. The file is committed because the web build needs it; run the script
+again after changing the presets or the crystal code (`--check` compares with
+the committed file). The settings are listed at the top of
+`js/diffractionsim-web/index.ts` and use the trait names; the web build
+defaults to CBED with Kikuchi lines on. A MyST page loads the same module with
+the `anywidget` directive:
+
+```text
+:::{anywidget} diffraction-sim.js
+{"preset": "Si (diamond cubic)", "zone_axis": [1, 1, 0], "thickness_A": 400}
+:::
+```

docs/api/html-export.md

@@ -88,6 +88,7 @@ the kernel's working directory instead, as `export_html()` does.
 | Show4DSTEM | `dtype="uint8"`, `"uint16"` | `detector_bin`, `scan_bin`: `1`, `2`, `4`, `8` | mean bin; `uint8` clips counts above 255 and says how many; `export_kind="interactive"` packs the counts into the page and refuses above 2 GiB, naming the least binning that fits; `export_kind="report"` writes static PNG virtual-image pages with no raw 4D |
 | ShowDiffraction | `full` (float32) | none | exact float32 frames |
 | ShowPDF | `full`, `uint8` | none | `full` stores the float32 curves; `uint8` adds every scan position's radial profile (8 bits per k bin) so mask, probe and line selections recompute in the browser |
+| DiffractionSim | `full` | none | no image data; embeds the crystal reflection list, the settings and the Kikuchi reference when computed; `presets=` embeds more structures for the crystal menu |
 
 ## Single-file exports
 

docs/api/index.md

@@ -51,5 +51,6 @@ readers and test agents.
 | [ShowPDF](showpdf) | `quantem.widget.showpdf.ShowPDF` | state JSON, results zip (CSV), PNG/PDF/TIFF curves, interactive HTML (`encoding="full"` / `encoding="uint8"`) |
 | [ChooseLattice](choose-lattice) | `quantem.widget.choose_lattice.ChooseLattice` | state JSON (`save_state=True`) |
 | [ShowCIF](showcif) | `quantem.widget.ShowCIF` | Interactive HTML with embedded coordinates |
+| [DiffractionSim](diffractionsim) | `quantem.widget.DiffractionSim` | state JSON, PNG, interactive HTML |
 
 All widget-level HTML exports follow the [HTML export](html-export) protocol.