From c6c3613bfdc58f016bd446f7ec1e43a51d7b93a5 Mon Sep 17 00:00:00 2001 From: Caden Myers Date: Thu, 29 Oct 2026 15:12:26 -0400 Subject: [PATCH 1/6] skpkg: restructure readme to roughly scikit-package standards --- README.rst | 87 ++++++++++++++++-------------------------------------- 1 file changed, 25 insertions(+), 62 deletions(-) diff --git a/README.rst b/README.rst index ae0b114..13a2858 100644 --- a/README.rst +++ b/README.rst @@ -10,11 +10,14 @@ |Forge| |PR| -|CI| |Tracking| +|CI| |Codecov| |Tracking| .. |CI| image:: https://github.com/diffpy/libdiffpy/actions/workflows/matrix-and-codecov.yml/badge.svg :target: https://github.com/diffpy/libdiffpy/actions/workflows/matrix-and-codecov.yml +.. |Codecov| image:: https://codecov.io/gh/diffpy/libdiffpy/branch/main/graph/badge.svg + :target: https://codecov.io/gh/diffpy/libdiffpy + .. |Forge| image:: https://img.shields.io/conda/vn/conda-forge/libdiffpy :target: https://anaconda.org/conda-forge/libdiffpy @@ -76,74 +79,30 @@ The following creates and activates a new environment named ``libdiffpy_env`` :: conda create -n libdiffpy_env libdiffpy conda activate libdiffpy_env -When compiling with the conda version of libdiffpy it is essential to -specify the header path, library path and runtime path of the active conda -environment. To verify the installation, compile and run the included -test code `examples/testlib.cpp `_ :: +To verify the installation, compile and run the included test code +`examples/testlib.cpp `_ :: cd examples c++ testlib.cpp -I$CONDA_PREFIX/include -L$CONDA_PREFIX/lib -Wl,-rpath,$CONDA_PREFIX/lib -ldiffpy ./a.out -Note the conda package for libdiffpy is built with C++ compilers -provided by conda-forge. This may cause incompatibility with system C++. +The conda package for libdiffpy is built with C++ compilers provided by +conda-forge, which may be incompatible with the system C++ compiler. In such case please use the conda-forge C++ compiler to link with libdiffpy. -Requirements -^^^^^^^^^^^^ - -To build libdiffpy from sources, a C++ compiler and the following software are required: - -* ``scons`` - software constructions tool (1.0 or later) -* ``libboost-dev`` - Boost C++ libraries development files (1.56 or later) -* ``GSL`` - GNU Scientific Library for C - -Recommended software: - -* ``libobjcryst`` - C++ library for free objects for crystallography, - https://github.com/diffpy/libobjcryst -* ``cxxtest`` - CxxTest Unit Testing Framework, http://cxxtest.com - -The required software is commonly available in the system package manager. -For example, on Ubuntu Linux the required software can be installed using :: - - sudo apt-get install build-essential scons libboost-dev libgsl0-dev - -Alternatively, all of the above can be installed into a conda environment from -the ``requirements`` folder :: +If you prefer to install from sources, obtain the source archive from +`GitHub `_ and ``cd`` into your +``libdiffpy`` directory. Create a conda environment with the build +dependencies listed in the ``requirements`` folder :: conda create -n libdiffpy_env --file requirements/build.txt --file requirements/conda.txt --file requirements/tests.txt conda activate libdiffpy_env -Installation from sources -^^^^^^^^^^^^^^^^^^^^^^^^^ - -Use sources from the git repository or extract the latest source -bundle from https://github.com/diffpy/libdiffpy/releases/latest. :: +Then build and install libdiffpy into the active conda environment :: - tar xzf libdiffpy-VERSION.tar.gz - cd libdiffpy-VERSION + scons -j4 install -To build and install the libdiffpy library use :: - - sudo scons -j4 install - -This installs libdiffpy for all users under the ``/usr/local`` directory. -If administrator (root) access is not available, see the output from -``scons --help`` for options to install to a user-writable location. - -To verify the libdiffpy installation, compile and run the included -test code `examples/testlib.cpp `_ :: - - cd examples - c++ testlib.cpp -ldiffpy - ./a.out - -If compilation fails because of missing header files or missing libdiffpy -library, adjust the ``CPATH`` and ``LIBRARY_PATH`` environment variables or -use the ``-I`` or ``-L`` compiler options. If the shared library libdiffpy -is unavailable at runtime, add a ``-Wl,-rpath,SomePath`` option to the -c++ command or adjust the ``LD_LIBRARY_PATH`` environment variable. +See the output from ``scons --help`` for the available build targets and options. Getting Started --------------- @@ -161,11 +120,14 @@ library with debug information and C-assertions checks :: scons -j4 build=develop +The library integrity can be verified by executing the unit tests :: + + scons -j4 test + The build script checks for a presence of ``sconsvars.py`` file, which can be used to permanently set the ``build`` variable. The SCons construction environment can be further customized in a ``sconscript.local`` -script. The library integrity can be verified by executing unit tests with -``scons test`` (requires the CxxTest framework). +script. See also the `instructions for compiling and modifying source code in development mode `_. @@ -176,10 +138,11 @@ hooks. 2. Initialize pre-commit (one time only) ``pre-commit install``. -Thereafter your code will be linted by black and isort and checked against flake8 before you can commit. -If it fails by black or isort, just rerun and it should pass (black and isort will modify the files so should -pass after they are modified). If the flake8 test fails please see the error messages and fix them manually before -trying to commit again. +Thereafter C++ sources will be formatted by clang-format, Python build scripts will be linted by black and +isort and checked against flake8, and spelling will be checked by codespell before you can commit. +If it fails by clang-format, black or isort, just rerun and it should pass (these hooks modify the files so +should pass after they are modified). If the flake8 or codespell checks fail please see the error messages and +fix them manually before trying to commit again. Improvements and fixes are always appreciated. From f17f096a37d7e9afbd5ca2ad444d689bf50ff7ef Mon Sep 17 00:00:00 2001 From: Caden Myers Date: Thu, 29 Oct 2026 15:19:38 -0400 Subject: [PATCH 2/6] doc: move dev instructions to rst format --- docs/development_instructions.md | 242 ------------------------------- docs/source/development.rst | 98 +++++++++++++ 2 files changed, 98 insertions(+), 242 deletions(-) delete mode 100644 docs/development_instructions.md create mode 100644 docs/source/development.rst diff --git a/docs/development_instructions.md b/docs/development_instructions.md deleted file mode 100644 index c609b24..0000000 --- a/docs/development_instructions.md +++ /dev/null @@ -1,242 +0,0 @@ -# Compile and Modify diffpy Source Code - ---- - -## I. Prerequisites - -### 1) Install System Dependencies - -Open a terminal and run the following commands in order: - -```bash -# Update package list -sudo apt update - -# Install build toolchain, CMake, Git, Python development headers, etc. -sudo apt install -y build-essential cmake git python3-dev - -# Install SCons (used to build libdiffpy) -sudo apt install -y scons - -# Install GSL (GNU Scientific Library) and ObjCryst (X-ray crystallography library) -sudo apt install -y libgsl-dev libobjcryst-dev -``` - ---- - -### 2) Create a Conda Virtual Environment - -> If you haven't installed Miniconda, please download and install it first - -```bash -# Create your environment (e.g., named your_env_name) with Python 3.11 -conda create -n your_env_name python=3.11 -y - -# Activate the environment (shell prompt will show (your_env_name)) -conda activate your_env_name - -# Install Python dependencies -conda install -y numpy - -# Install additional scientific libraries -conda install -c conda-forge boost-cpp gsl pyobjcryst -``` - -All subsequent operations should be performed within the (your_env_name) environment! - ---- - -## II. Get the Source Code - -Assume our working directory is `~/workspace` (adjust to your actual setup): - -```bash -# Switch to the working directory -cd ~/workspace - -# Clone the two core repositories -git clone https://github.com/diffpy/libdiffpy.git diffpy.libdiffpy -git clone https://github.com/diffpy/diffpy.srreal.git diffpy.srreal -``` - -> The directory structure should be: `~/workspace/` -> -> ``` -> ~/workspace/ -> ├── diffpy.libdiffpy/ ← C++ core library -> └── diffpy.srreal/ ← Python interface layer -> ``` - ---- - -## III. Initial Build (Developer Mode) - -> We use `--inplace` so after modifying code you only need to recompile, no `pip install` required. - ---- - -### Step 1: Build libdiffpy (C++ Library) - -```bash -# Enter the libdiffpy directory -cd ~/workspace/diffpy.libdiffpy - -# Clean old builds (optional but recommended) -rm -rf build/ - -# Build the C++ library with SCons -scons -``` - -Success indicators: - -- `build/fast-x86_64/libdiffpy.so` is generated -- No errors in terminal output; warnings can be ignored - -Verify it was generated: - -```bash -ls -l build/fast-x86_64/libdiffpy.so -``` - -> If you see `ObjCryst not found` or `gsl-config not found`, make sure you ran the `apt install` commands in Step 1. - ---- - -### Step 2: Set Environment Variables - -To allow srreal to find the newly built `libdiffpy.so`, set the following variables and replace the paths with your actual paths: - -```bash -# Set the libdiffpy build path -export DIFFPY_LIBDIFFPY_BUILD="$HOME/workspace/diffpy.libdiffpy/build/fast-x86_64" - -# Tell the linker and runtime where to find .so files -export LD_LIBRARY_PATH="$DIFFPY_LIBDIFFPY_BUILD:$LD_LIBRARY_PATH" -export LIBRARY_PATH="$DIFFPY_LIBDIFFPY_BUILD:$LIBRARY_PATH" - -# Let Python import srreal source directly (developer mode) -export PYTHONPATH="$HOME/workspace/diffpy.srreal/src:$PYTHONPATH" -``` - ---- - -### Step 3: Build diffpy.srreal (Python Extension) - -```bash -# Enter the srreal directory -cd ~/workspace/diffpy.srreal - -# Clean old builds -rm -rf build/ - -# Build the Python extension (generate .so in place) -python setup.py build_ext --inplace -``` - -Success indicators: - -- `diffpy/srreal/srreal_ext.cpython-*.so` is generated -- No `cannot find -ldiffpy` errors - ---- - -### Step 4: Verify the Installation - -```bash -# Test whether it can be imported -python -c "from diffpy.srreal.pdfcalculator import PDFCalculator; print('srreal Success！')" -``` - -If you see `srreal Success！`, everything is working! - ---- - -## IV. Daily Development: Modify Code + Rebuild - -### Scenario A: Only Modified diffpy.srreal Python or C++ Code - -```bash -# 1. Activate your Conda environment -conda activate your_env_name - -# 2. Enter the srreal directory -cd ~/workspace/diffpy.srreal - -# 3. Clean and rebuild -rm -rf build/ -python setup.py build_ext --inplace -``` - -No need to rebuild libdiffpy - ---- - -### Scenario B: Modified diffpy.libdiffpy C++ Code - -```bash -# 1. Activate your Conda environment -conda activate your_env_name - -# 2. Rebuild libdiffpy -cd ~/workspace/diffpy.libdiffpy -rm -rf build/ -scons - -# 3. Update environment variables (ensure they point to the latest .so) -export DIFFPY_LIBDIFFPY_BUILD="$HOME/workspace/diffpy.libdiffpy/build/fast-x86_64" -export LD_LIBRARY_PATH="$DIFFPY_LIBDIFFPY_BUILD:$LD_LIBRARY_PATH" - -# 4. Rebuild srreal (since it depends on libdiffpy) -cd ~/workspace/diffpy.srreal -rm -rf build/ -python setup.py build_ext --inplace -``` - -Key point: if you change the underlying C++ library, you must rebuild the Python extension - ---- - -## V. Common Issues and Fixes - -### Issue 1: ModuleNotFoundError: No module named 'numpy' - -→ Fix: run `conda install numpy` in your Conda environment - -### Issue 2: cannot find -ldiffpy - -→ Fix: - -1. Confirm `libdiffpy.so` has been generated -2. Confirm `LD_LIBRARY_PATH` and `LIBRARY_PATH` are set correctly - -### Issue 3: gsl-config: not found - -→ Fix: run `sudo apt install libgsl-dev` - -### Issue 4: Checking for C++ library ObjCryst... no - -→ Fix: run `sudo apt install libobjcryst-dev` - -If you encounter other issues, you can search relevant forums or contact qiaohai@tongji.edu.cn - ---- - -## VI. Appendix: One-Click Environment Variable Setup - -You can append the following to `~/.bashrc` so you don't have to set environment variables every time: - -```bash -# diffpy development environment -export DIFFPY_WORKSPACE="$HOME/workspace" -export DIFFPY_LIBDIFFPY_BUILD="$DIFFPY_WORKSPACE/diffpy.libdiffpy/build/fast-x86_64" -export LD_LIBRARY_PATH="$DIFFPY_LIBDIFFPY_BUILD:$LD_LIBRARY_PATH" -export LIBRARY_PATH="$DIFFPY_LIBDIFFPY_BUILD:$LIBRARY_PATH" -export PYTHONPATH="$DIFFPY_WORKSPACE/diffpy.srreal/src:$PYTHONPATH" -``` - -Then run: - -```bash -source ~/.bashrc -``` diff --git a/docs/source/development.rst b/docs/source/development.rst new file mode 100644 index 0000000..602ff55 --- /dev/null +++ b/docs/source/development.rst @@ -0,0 +1,98 @@ +:tocdepth: -1 + +.. index:: development + +================= +Development guide +================= + +This page describes how to build and test libdiffpy from sources when +modifying the library. See the +`README `_ +for how to fork the repository and set up the pre-commit hooks. + +Set up the build environment +---------------------------- + +From the root of your libdiffpy clone, create a conda environment with the +build and test dependencies listed in the ``requirements`` folder :: + + conda create -n libdiffpy_env --file requirements/build.txt --file requirements/conda.txt --file requirements/tests.txt + conda activate libdiffpy_env + +Build the library +----------------- + +Build the shared library with :: + + scons -j4 + +The ``build`` option selects the compiler settings, for example +``scons -j4 build=develop``. On Linux and macOS the build variants are + +* ``fast`` - optimized build with C assertion checks disabled (default) +* ``debug`` - build with debug information and C assertion checks +* ``develop`` - unoptimized build with C assertion checks +* ``coverage`` - unoptimized build instrumented for coverage analysis + +The library is built in the ``build/-`` directory, +for example ``build/fast-x86_64/``. Run ``scons --help`` for the full list +of build targets and options. + +To use the same options for every build, set them in a ``sconsvars.py`` +file in the repository root, for example :: + + build = 'develop' + +The SCons construction environment can be further customized in a +``sconscript.local`` file in the repository root. Both files are ignored +by git. For example, the following ``sconscript.local`` builds with the +conda-forge compiler and records the conda library directory as the runtime +path, so that the unit tests find the conda libraries :: + + Import('env') + import os + prefix = os.environ['CONDA_PREFIX'] + env.Replace(CXX=os.path.join(prefix, 'bin', 'c++')) + env.Prepend(CCFLAGS=['-isystem' + os.path.join(prefix, 'include')]) + env.Append(LIBPATH=os.path.join(prefix, 'lib')) + env.Append(LINKFLAGS='-Wl,-rpath,' + os.path.join(prefix, 'lib')) + +Run the unit tests +------------------ + +The unit tests use the CxxTest framework. Build and run all of them with :: + + scons -j4 test + +To run only some of the tests, pass fixed-string patterns of the test file +names with the ``tests`` option. Patterns are case-insensitive and can be +separated by commas :: + + scons -j4 test tests=PDFCalculator,Lattice + +To run the tests against the installed library instead of the one in the +``build`` directory, use ``scons test test_installed=True``. + +Measure test coverage +--------------------- + +Build with the ``coverage`` variant, run the tests and summarize the results +with ``gcovr`` :: + + scons -j4 build=coverage test + gcovr --root . --filter 'build/coverage-.*/diffpy/' + +Use ``scons zerocounters`` to reset the coverage counts between runs. + +Test changes with diffpy.srreal +------------------------------- + +`diffpy.srreal `_ compiles the +libdiffpy sources from the commit pinned in its +``cmake/LibdiffpyVersion.cmake`` file. To build diffpy.srreal against a local +libdiffpy clone, point the CMake ``FETCHCONTENT_SOURCE_DIR_LIBDIFFPY`` +variable to it. diffpy.srreal checks that the clone is at the pinned commit, +so update the pin in ``cmake/LibdiffpyVersion.cmake`` when testing new +libdiffpy commits. See the diffpy.srreal documentation for its build +instructions. From cc7e5ca2ec54b7837bf26497c7a9940c09879c0c Mon Sep 17 00:00:00 2001 From: Caden Myers Date: Thu, 29 Oct 2026 15:20:00 -0400 Subject: [PATCH 3/6] doc: add 1.4.1 release to the CHANGELOG --- CHANGELOG.rst | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 44e268d..0c8cb09 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -4,6 +4,38 @@ Release notes .. current developments +1.4.1 +===== + +Released 2025-06-25. Notable differences from version 1.4.0. + +**Added:** + +* ``qbroad_seperable`` attribute to ``JeongPeakWidth`` for a Q-dependent + peak broadening term, see https://doi.org/10.1107/S0108767391011327. +* Build scripts for Windows. +* GitHub release with the source tarball SHA256 created automatically + after a tag push. + +**Changed:** + +* Require libobjcryst 2017.2.1 or later. +* Add the conda environment include and library paths to the build when + ``CONDA_PREFIX`` is set. +* Do not create ``OverlapCalculator`` self-neighborhoods at sites excluded + by the pair mask. + +**Removed:** + +* Runtime check for the ObjCryst inversion center bug, which is fixed in + libobjcryst 2017.2.1. + +**Fixed:** + +* ``OverlapCalculator`` now respects its ``rmin`` value. +* Build with ``REAL=double`` to match the floating-point type of libobjcryst. +* Read the installation prefix from the ``PREFIX`` environment variable. + 1.4.0 ===== From 54a61a3240c32abe1cb9e40b78637a8cf46dd632 Mon Sep 17 00:00:00 2001 From: Caden Myers Date: Thu, 29 Oct 2026 15:20:37 -0400 Subject: [PATCH 4/6] doc: update release checklist to be more accurate --- .github/ISSUE_TEMPLATE/release_checklist.md | 2 +- README.rst | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/release_checklist.md b/.github/ISSUE_TEMPLATE/release_checklist.md index 63e624b..bc397e2 100644 --- a/.github/ISSUE_TEMPLATE/release_checklist.md +++ b/.github/ISSUE_TEMPLATE/release_checklist.md @@ -16,7 +16,7 @@ assignees: "" - [ ] Grammar and writing quality are checked (no typos). - [ ] `FALLBACK_VERSION` in `site_scons/fallback_version.py` is updated to `.post0`. Without this, the SCons build fails after tagging. - [ ] `scons test` passes locally with the latest conda-forge compilers and dependencies. -- [ ] Dispatch matrix testing to test the release on all systems. If you do not have permission to run this workflow, tag the maintainer and say `@maintainer, please dispatch matrix testing workflow`. Matrix testing also runs automatically on the tag and blocks the release if it fails. +- [ ] Dispatch matrix testing to test the release on all systems. If you do not have permission to run this workflow, tag the maintainer and say `@maintainer, please dispatch matrix testing workflow`. Matrix testing also runs automatically after a GitHub release is published, but it does not block the release, so check that it passes. Please tag the maintainer (e.g., @username) in the comment here when you are ready for the GitHub release. Include any additional comments necessary, such as version information and details about the pre-release here: diff --git a/README.rst b/README.rst index 13a2858..2237552 100644 --- a/README.rst +++ b/README.rst @@ -116,7 +116,7 @@ If you see a bug or want to request a feature, please `report it as an issue `_. +See also the `development guide `_ for build variants, running a subset of +the unit tests, coverage analysis and testing changes with diffpy.srreal. To ensure code quality and to prevent accidental commits into the default branch, please set up the use of our pre-commit hooks. From 17700a1836da5340ae6b278dd67e60742c97a852 Mon Sep 17 00:00:00 2001 From: Caden Myers Date: Thu, 29 Oct 2026 15:23:09 -0400 Subject: [PATCH 5/6] doc: rm unused import from docs.txt and other minor doc changes --- docs/source/conf.py | 2 +- docs/source/index.rst | 1 + requirements/docs.txt | 1 - 3 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/conf.py b/docs/source/conf.py index 784ff33..2da4f39 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -305,7 +305,7 @@ "libdiffpy Documentation", ab_authors, "libdiffpy", - "One line description of project.", + "DiffPy C++ library for calculation of PDF and other real-space quantities", # noqa: E501 "Miscellaneous", ), ] diff --git a/docs/source/index.rst b/docs/source/index.rst index 4164f2d..fa49529 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -54,6 +54,7 @@ Table of contents .. toctree:: :maxdepth: 2 + development release license diff --git a/requirements/docs.txt b/requirements/docs.txt index 1de813f..9a46bd5 100644 --- a/requirements/docs.txt +++ b/requirements/docs.txt @@ -1,5 +1,4 @@ sphinx sphinx_rtd_theme sphinx-copybutton -doctr m2r2 From 79d75836dea7e688875f5285c577e808f304d55d Mon Sep 17 00:00:00 2001 From: Caden Myers Date: Thu, 29 Oct 2026 15:23:17 -0400 Subject: [PATCH 6/6] news --- news/easy-changes.rst | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 news/easy-changes.rst diff --git a/news/easy-changes.rst b/news/easy-changes.rst new file mode 100644 index 0000000..3cc923d --- /dev/null +++ b/news/easy-changes.rst @@ -0,0 +1,26 @@ +**Added:** + +* Codecov badge in the README. +* Development guide in the documentation for build variants, running a subset of the unit tests, coverage analysis and testing changes with ``diffpy.srreal``. +* Release notes for version 1.4.1 in ``CHANGELOG.rst``. + +**Changed:** + +* Restructure the README to follow the scikit-package template and point to the ``requirements`` folder instead of listing dependencies. +* Describe the clang-format and codespell pre-commit hooks in the README. + +**Deprecated:** + +* + +**Removed:** + +* Outdated ``docs/development_instructions.md``, replaced by the development guide. + +**Fixed:** + +* + +**Security:** + +*