sync build system choices and approaches with pyobjcryst

sbillinge · Oct 8, 2026 · Jun 25, 2026 · Jun 26, 2026 · Jun 26, 2026 · Jun 24, 2026
commit 25f58b04f35e7fa85e8f4c84e065d4636e133983
diff --git a/.github/workflows/_build-and-test.yml b/.github/workflows/_build-and-test.yml
@@ -0,0 +1,90 @@
+name: Build and test the nanobind distribution
+
+on:
+  workflow_call:
+    secrets:
+      CODECOV_TOKEN:
+        required: false
+
+jobs:
+  wheels:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-24.04, macos-14]
+        python: ["3.12", "3.13", "3.14"]
+    defaults:
+      run:
+        shell: bash -l {0}
+    env:
+      CMAKE_BUILD_PARALLEL_LEVEL: "2"
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+          submodules: recursive
+      - uses: conda-incubator/setup-miniconda@v3
+        with:
+          activate-environment: build
+          channels: conda-forge
+          auto-activate-base: false
+          python-version: ${{ matrix.python }}
+      - name: Install build and test dependencies
+        run: |
+          conda install -y --file requirements/conda.txt
+          python -m pip install build pytest pytest-cov
+      - name: Build sdist and then wheel from the sdist
+        run: |
+          # Check the revision before the sdist discards Git metadata.
+          cmake -DLIBDIFFPY_SOURCE_DIR="$PWD/extern/libdiffpy" -P cmake/LibdiffpyVersion.cmake
+          python -m build
+      - name: Bundle Linux runtime libraries
+        if: runner.os == 'Linux'
+        run: |
+          python -m pip install auditwheel patchelf
+          LD_LIBRARY_PATH="$CONDA_PREFIX/lib" auditwheel repair dist/*.whl -w wheelhouse
+      - name: Bundle macOS runtime libraries
+        if: runner.os == 'macOS'
+        run: |
+          python -m pip install delocate
+          delocate-wheel -w wheelhouse dist/*.whl
+      - uses: actions/checkout@v6
+        with:
+          repository: diffpy/pyobjcryst
+          # Integration against the current migration, not an orphanable SHA.
+          # Switch to a release tag when the nanobind migration is released.
+          ref: nanobind_migration_test
+          submodules: recursive
+          path: .integration/pyobjcryst
+      - name: Record the pyobjcryst integration revision
+        run: |
+          revision=$(git -C .integration/pyobjcryst rev-parse HEAD)
+          echo "pyobjcryst nanobind_migration_test: $revision"
+          {
+            echo "### pyobjcryst integration source"
+            echo "The migration branch is a moving integration dependency."
+            echo "Resolved commit: https://github.com/diffpy/pyobjcryst/commit/$revision"
+            git -C .integration/pyobjcryst submodule status --recursive
+          } >> "$GITHUB_STEP_SUMMARY"
+      - name: Install pyobjcryst migration and the built wheel
+        run: |
+          python -m pip install ./.integration/pyobjcryst
+          python -m pip install wheelhouse/*.whl
+          python -c "import pyobjcryst._pyobjcryst"
+      - name: Test installed wheel and ObjCryst integration
+        run: python -m pytest tests --cov=diffpy.srreal --cov-report=xml
+      - uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: coverage.xml
+          fail_ci_if_error: false
+      - uses: actions/upload-artifact@v4
+        with:
+          name: wheels-${{ matrix.os }}-${{ matrix.python }}
+          path: wheelhouse/*.whl
+      - uses: actions/upload-artifact@v4
+        if: matrix.os == 'ubuntu-24.04' && matrix.python == '3.14'
+        with:
+          name: sdist
+          path: dist/*.tar.gz
diff --git a/.github/workflows/build-wheel-release-upload.yml b/.github/workflows/build-wheel-release-upload.yml
@@ -4,15 +4,47 @@ on:
   workflow_dispatch:
   push:
     tags:
-      - "*" # Trigger on all tags initially, but tag and release privilege are verified in _build-wheel-release-upload.yml
+      - "*"
+
+permissions:
+  contents: write
 
 jobs:
-  release:
-    uses: scikit-package/release-scripts/.github/workflows/_build-wheel-release-upload.yml@v0
+  check-tag-privilege:
+    uses: scikit-package/release-scripts/.github/workflows/_check-tag-privilege.yml@v0
     with:
-      project: diffpy.srreal
-      c_extension: true
-      maintainer_GITHUB_username: sbillinge
+      maintainer_github_username: sbillinge
+
+  check-pat-token:
+    uses: scikit-package/release-scripts/.github/workflows/_check-pat-token-validity.yml@v0
+    secrets:
+      PAT_TOKEN: ${{ secrets.PAT_TOKEN }}
+
+  check-pypi-token:
+    uses: scikit-package/release-scripts/.github/workflows/_check-pypi-token-validity.yml@v0
     secrets:
       PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
+
+  build:
+    needs: [check-tag-privilege, check-pat-token, check-pypi-token]
+    uses: ./.github/workflows/_build-and-test.yml
+    secrets:
+      CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
+
+  release-github:
+    needs: build
+    uses: scikit-package/release-scripts/.github/workflows/_release-github.yml@v0
+    secrets:
       PAT_TOKEN: ${{ secrets.PAT_TOKEN }}
+
+  release-pypi:
+    needs: release-github
+    uses: scikit-package/release-scripts/.github/workflows/_release-pypi.yml@v0
+    with:
+      python_version: 3.14
+    secrets:
+      PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
+
+  docs:
+    needs: release-github
+    uses: ./.github/workflows/publish-docs-on-release.yml
diff --git a/.github/workflows/matrix-and-codecov-on-merge-to-main.yml b/.github/workflows/matrix-and-codecov-on-merge-to-main.yml
@@ -12,10 +12,6 @@ on:
 
 jobs:
   matrix-coverage:
-    uses: scikit-package/release-scripts/.github/workflows/_matrix-and-codecov-on-merge-to-main.yml@v0
-    with:
-      project: diffpy.srreal
-      c_extension: true
-      headless: false
+    uses: ./.github/workflows/_build-and-test.yml
     secrets:
       CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
diff --git a/.github/workflows/publish-docs-on-release.yml b/.github/workflows/publish-docs-on-release.yml
@@ -2,11 +2,38 @@ name: Deploy Documentation on Release
 
 on:
   workflow_dispatch:
+  workflow_call:
+
+permissions:
+  contents: write
 
 jobs:
   docs:
-    uses: scikit-package/release-scripts/.github/workflows/_publish-docs-on-release.yml@v0
-    with:
-      project: diffpy.srreal
-      c_extension: true
-      headless: false
+    runs-on: ubuntu-24.04
+    defaults:
+      run:
+        shell: bash -l {0}
+    env:
+      CMAKE_BUILD_PARALLEL_LEVEL: "2"
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+          submodules: recursive
+      - uses: conda-incubator/setup-miniconda@v3
+        with:
+          activate-environment: docs
+          channels: conda-forge
+          auto-activate-base: false
+          python-version: "3.14"
+      - name: Install package and documentation dependencies
+        run: |
+          conda install -y --file requirements/conda.txt
+          python -m pip install -r requirements/docs.txt
+          python -m pip install -e .
+      - name: Build documentation
+        run: make -C docs html
+      - uses: peaceiris/actions-gh-pages@v4
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: ./docs/build/html
diff --git a/.github/workflows/tests-on-pr.yml b/.github/workflows/tests-on-pr.yml
@@ -6,10 +6,6 @@ on:
 
 jobs:
   tests-on-pr:
-    uses: scikit-package/release-scripts/.github/workflows/_tests-on-pr.yml@v0
-    with:
-      project: diffpy.srreal
-      c_extension: true
-      headless: false
+    uses: ./.github/workflows/_build-and-test.yml
     secrets:
       CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
diff --git a/.gitmodules b/.gitmodules
@@ -0,0 +1,3 @@
+[submodule "extern/libdiffpy"]
+	path = extern/libdiffpy
+	url = https://github.com/diffpy/libdiffpy.git
diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -0,0 +1,71 @@
+cmake_minimum_required(VERSION 3.20)
+project(diffpy_srreal LANGUAGES CXX)
+
+# Keep this aligned with the pinned libdiffpy's src/SConscript.
+set(CMAKE_CXX_STANDARD 23)
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+set(CMAKE_CXX_EXTENSIONS OFF)
+
+# Conda is optional; system packages and CMAKE_PREFIX_PATH work as well.
+if(DEFINED ENV{CONDA_PREFIX})
+    list(APPEND CMAKE_PREFIX_PATH "$ENV{CONDA_PREFIX}" "$ENV{CONDA_PREFIX}/Library")
+endif()
+
+find_package(Python 3.12 REQUIRED COMPONENTS Interpreter Development.Module NumPy)
+execute_process(
+    COMMAND "${Python_EXECUTABLE}" -m nanobind --cmake_dir
+    OUTPUT_VARIABLE nanobind_DIR
+    OUTPUT_STRIP_TRAILING_WHITESPACE
+    COMMAND_ERROR_IS_FATAL ANY
+)
+find_package(nanobind CONFIG REQUIRED)
+find_package(Boost CONFIG REQUIRED COMPONENTS serialization)
+find_package(GSL REQUIRED)
+
+set(LIBDIFFPY_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/extern/libdiffpy")
+if(NOT EXISTS "${LIBDIFFPY_SOURCE_DIR}/src/diffpy/version.tpl")
+    message(FATAL_ERROR "Missing libdiffpy sources. Run git submodule update --init --recursive.")
+endif()
+
+include(cmake/LibdiffpyVersion.cmake)
+configure_file("${LIBDIFFPY_SOURCE_DIR}/src/diffpy/version.tpl"
+               "include/diffpy/version.hpp")
+set(DIFFPY_HAS_OBJCRYST 0)
+configure_file("${LIBDIFFPY_SOURCE_DIR}/src/diffpy/features.tpl"
+               "include/diffpy/features.hpp")
+
+file(GLOB LIBDIFFPY_SOURCES CONFIGURE_DEPENDS
+    "${LIBDIFFPY_SOURCE_DIR}/src/diffpy/*.cpp"
+    "${LIBDIFFPY_SOURCE_DIR}/src/diffpy/srreal/*.cpp"
+)
+# ObjCryst conversion uses the public Python API, so it does not depend on
+# pyobjcryst's private static library, compiler, REAL precision, or nanobind ABI.
+list(FILTER LIBDIFFPY_SOURCES EXCLUDE REGEX "ObjCryst")
+# Include every object: libdiffpy has static factory/serialization registrars
+# that an ordinary static archive link can otherwise discard.
+add_library(diffpy_core OBJECT ${LIBDIFFPY_SOURCES})
+set_target_properties(diffpy_core PROPERTIES
+    POSITION_INDEPENDENT_CODE ON
+    CXX_VISIBILITY_PRESET hidden
+    VISIBILITY_INLINES_HIDDEN ON
+)
+target_include_directories(diffpy_core PUBLIC
+    "${CMAKE_CURRENT_BINARY_DIR}/include"
+    "${LIBDIFFPY_SOURCE_DIR}/src"
+)
+target_compile_definitions(diffpy_core PRIVATE DIFFPYRUNTIMERELPATH=data)
+target_link_libraries(diffpy_core PUBLIC Boost::serialization GSL::gsl GSL::gslcblas ${CMAKE_DL_LIBS})
+
+file(GLOB SRREAL_SOURCES CONFIGURE_DEPENDS "src/extensions/*.cpp")
+# Keep the CPython-specific ABI while the bindings use the NumPy C API and
+# CPython macros. Do not label these wheels abi3 without porting those calls.
+nanobind_add_module(srreal_ext NB_STATIC ${SRREAL_SOURCES})
+target_link_libraries(srreal_ext PRIVATE diffpy_core Python::NumPy)
+set_target_properties(srreal_ext PROPERTIES INSTALL_RPATH_USE_LINK_PATH ON)
+install(TARGETS srreal_ext LIBRARY DESTINATION diffpy/srreal)
+install(DIRECTORY "${LIBDIFFPY_SOURCE_DIR}/src/runtime/"
+        DESTINATION diffpy/srreal/data
+        FILES_MATCHING PATTERN "*.dat" PATTERN "bvparm2011sel.cif")
+install(FILES "${LIBDIFFPY_SOURCE_DIR}/LICENSE.txt"
+              "${LIBDIFFPY_SOURCE_DIR}/LICENSE_DANSE.txt"
+        DESTINATION diffpy/srreal/licenses/libdiffpy)
diff --git a/MANIFEST.in b/MANIFEST.in
diff --git a/cmake/LibdiffpyVersion.cmake b/cmake/LibdiffpyVersion.cmake
@@ -0,0 +1,13 @@
+# Metadata for the pinned extern/libdiffpy gitlink. Keep this in sync when
+# updating the submodule; source distributions must also build without Git.
+set(DIFFPY_VERSION_MAJOR 1)
+set(DIFFPY_VERSION_MINOR 4)
+set(DIFFPY_VERSION_MICRO 0)
+set(DIFFPY_VERSION_PATCH 0)
+set(DIFFPY_VERSION 1004000000LL)
+set(DIFFPY_VERSION_STR "1.4.0")
+set(DIFFPY_VERSION_DATE "2026-06-24 00:40:51 -0600")
+set(DIFFPY_GIT_SHA "b191c66df059aa1e4f96260b9df483454f32e084")
+
+include("${CMAKE_CURRENT_LIST_DIR}/VerifyLibdiffpyRevision.cmake")
+verify_libdiffpy_revision("${LIBDIFFPY_SOURCE_DIR}" "${DIFFPY_GIT_SHA}")
diff --git a/cmake/VerifyLibdiffpyRevision.cmake b/cmake/VerifyLibdiffpyRevision.cmake
@@ -0,0 +1,27 @@
+# Source archives deliberately have no .git entry. Test for one before running
+# Git so an archive unpacked inside another repository cannot inherit its HEAD.
+function(verify_libdiffpy_revision source_dir expected_revision)
+    if(NOT EXISTS "${source_dir}/.git")
+        return()
+    endif()
+
+    find_package(Git REQUIRED)
+    execute_process(
+        COMMAND "${GIT_EXECUTABLE}" -C "${source_dir}" rev-parse --verify HEAD
+        RESULT_VARIABLE git_result
+        OUTPUT_VARIABLE actual_revision
+        OUTPUT_STRIP_TRAILING_WHITESPACE
+        ERROR_VARIABLE git_error
+    )
+    if(NOT git_result EQUAL 0)
+        message(FATAL_ERROR "Cannot determine the libdiffpy revision: ${git_error}")
+    endif()
+    if(NOT actual_revision STREQUAL expected_revision)
+        message(FATAL_ERROR
+            "libdiffpy revision mismatch: cmake/LibdiffpyVersion.cmake records "
+            "${expected_revision}, but ${source_dir} is at ${actual_revision}. "
+            "Run git submodule update --init --recursive to restore the pin, "
+            "or update LibdiffpyVersion.cmake together with the submodule."
+        )
+    endif()
+endfunction()
diff --git a/devutils/pyobjcryst-101-crash-report.md b/devutils/pyobjcryst-101-crash-report.md
@@ -0,0 +1,42 @@
+The nanobind migration still has an unsafe return path in `Crystal.GetScatteringPowerRegistry().GetObj(i)`. This surfaced while integrating the pinned branch with diffpy.srreal. It appears to be the **return-direction virtual-inheritance problem** described in the migration notes.
+
+Reproduced on `117cb30a86dc64d0cb455cf85b53d3b269b61e65` (ObjCryst++ `30fc8df`), Linux x86_64, Python 3.14.6, nanobind 3.1.0, NumPy 2.5.1; extension built with GCC 16.2.1.
+
+Self-contained reproducer (no diffpy.srreal import needed):
+
+```python
+from io import StringIO
+from pyobjcryst.crystal import create_crystal_from_cif
+
+crystal = create_crystal_from_cif(StringIO("""data_test
+_cell_length_a 4
+_cell_length_b 4
+_cell_length_c 4
+_cell_angle_alpha 90
+_cell_angle_beta 90
+_cell_angle_gamma 90
+_symmetry_space_group_name_H-M 'P 1'
+loop_
+_atom_site_label
+_atom_site_type_symbol
+_atom_site_fract_x
+_atom_site_fract_y
+_atom_site_fract_z
+C1 C 0 0 0
+"""))
+
+registry = crystal.GetScatteringPowerRegistry()
+sp = registry.GetObj(0)
+sp.SetBiso(0.5)
+print(crystal.GetScatteringComponentList()[0].mpScattPow.GetBiso(),
+      flush=True)  # prints 0.0, not the requested 0.5
+print(sp.GetName(), flush=True)  # SIGSEGV
+```
+
+Running this with `python -X faulthandler` exits with SIGSEGV (subprocess return code -11). In this minimal case `SetBiso` itself returns but fails to update the actual scattering power; the subsequent `GetName` crashes. The same registry/`SetBiso` pattern has also been reported to segfault directly in downstream testing. These should not be treated as harmless shutdown leak diagnostics.
+
+The generic registry bindings in [nb_objregistry.cpp](https://github.com/diffpy/pyobjcryst/blob/117cb30a86dc64d0cb455cf85b53d3b269b61e65/src/extensions/nb_objregistry.cpp#L60) return a base-typed `ScatteringPower&` with `reference_internal`. Unlike the explicit concrete-type handling in `nb_crystal.cpp` and `nb_scatteringcomponent.cpp`, this lets nanobind attempt the unsafe virtual-base adjustment for `ScatteringPowerAtom`.
+
+Replacing just `sp = registry.GetObj(0)` with `sp = crystal.GetScatteringComponentList()[0].mpScattPow` yields the expected Biso of 0.5. We use that accessor as a temporary workaround in diffpy.srreal's PDF tests, keeping the original reference data and tolerances. It does not protect srfit or user scripts that use the registry pattern.
+
+Suggested fix: apply the same explicit `dynamic_cast` to the concrete scattering-power type before `nb::cast(..., reference_internal, parent)` for registry returns, and audit the other `GetObj` overloads, indexing, and iteration for the same issue. A regression test should create scattering powers through CIF loading, access them through the registry, and verify both mutation visibility through the crystal and safe method calls.
diff --git a/extern/libdiffpy b/extern/libdiffpy