Missing cblas.dll in the Windows pypi wheel

Problem

diffpy.pdffit2 installed on windows (any Python) can't be run, since gsl-25.dll requires libcblas.dll to run.

create new python environment
pip install diffpy.pdffit2
run python -c "import diffpy.pdffit2"
observe import error

Looking at gsl-25.dll included in the wheel, we see dependency on libcblas.dll

Proposed solution

Add libcblas.dll (and any potential dependencies of it) to the wheel. This can be taken from the conda-forge distribution of libcblas

@Tieqiong

@Tieqiong this seems familiar, but please could you take a look?

@sbillinge

@sbillinge Yes will test and fix this.

@sbillinge

@sbillinge Have we fixed this? Since I don't have laptop with Windows at hand, I cannot test it right away.

I will check. Please give me a minute

I can confirm that everything works when installing from conda-forge, however, when installing directly from pip we are getting this import error:

I can confirm that everything works when installing from conda-forge, however, when installing directly from pip we are getting this import error:

I'm looking into this and checking where it breaks the import.

@Tieqiong

Thanks. If you need help, please reach out to @Tieqiong

@Tieqiong

@Tieqiong could you help to look at why now we cannot import the pdffit2 directly by pip install it?

@stevenhua0320

@stevenhua0320 I checked and this is still not working on our latest vesrsion of the code.

Here are steps to fix this per chatgpt:
Here’s the practical, step-by-step way to bundle a C library DLL (e.g., libcblas.dll) inside a Python wheel so users don’t have to install it separately — exactly what the advice you received is referring to.

✅ How Python wheels handle DLLs on Windows

Wheels can contain native shared libraries (.dll, .so, .dylib) inside the wheel.
At runtime, Python loads them automatically as long as they are placed in the correct location in the wheel.

For Windows, the correct place is usually:

mypackage/
    ___init__.py
    your_python_modules.py
    libcblas.dll     ← place DLL directly inside your package

Python will automatically search for DLLs located next to your extension modules or package code.

✅ How to include `libcblas.dll` in your wheel

Step 1 — Get `libcblas.dll` from conda-forge

Install the conda-forge package:

micromamba install -c conda-forge libcblas

Then locate the DLL:

<micromamba env>/Library/bin/libcblas.dll

Copy that file into your Python package directory, e.g.:

yourpackage/
    __init__.py
    ...
    lib/
        libcblas.dll

Or put it at the top level of the package:

yourpackage/libcblas.dll

Both layouts work, but top-level inside the package is simplest.

✅ Step 2 — Add the DLL to your wheel using `package_data`

If you're using pyproject.toml with setuptools, add something like:

[tool.setuptools.package-data]
yourpackage = ["libcblas.dll"]

Or if you put it into a lib/ folder:

[tool.setuptools.package-data]
yourpackage = ["lib/*.dll"]

If you're using setup.py, use package_data:

from setuptools import setup

setup(
    name="yourpackage",
    ...
    package_data={
        "yourpackage": ["libcblas.dll"],   # Or "lib/*.dll"
    },
    include_package_data=True,
)

✅ Step 3 — Verify that the wheel contains the DLL

Build your wheel:

python -m build

List the wheel contents:

unzip -l dist/yourpackage-*.whl

You should see:

yourpackage/libcblas.dll

✅ Step 4 — Ensure Python can load the DLL

Put this in your __init__.py or in your module that loads C code:

import os
import sys

# Add package directory to PATH so Windows can find the DLL
this_dir = os.path.dirname(__file__)
os.add_dll_directory(this_dir)

Python 3.8+ needs this on Windows because DLL loading rules changed.

If your DLL is in a subfolder:

os.add_dll_directory(os.path.join(this_dir, "lib"))

⭐ Extra Notes

1. Bundling dependencies of dependencies

libcblas.dll may depend on:

libgfortran
libgcc
msvcp???.dll
etc.

You can discover dependencies using:

objdump -p libcblas.dll  # On Linux

or:

dumpbin /DEPENDENTS libcblas.dll

Most conda-forge DLLs depend on other conda-forge DLLs — you will need to include those as well unless they are Windows system DLLs.

@sbillinge

@sbillinge I have checked that libcblas file would have different file format. For MacOS (since I'm in Mac), the file ends with .dylib, which is not the file with format dll. I think dll file is supported on windows only. So, to get the file I think I need to manually download the dll file from conda-forge?
From my point, it is safer to do it for a person with a windows system as it can directly download from conda-forge with correct dll format and can build the package locally to see whether there's something wrong in the build of the package.

Sounds good. Please work directly with someone who has windows.

@stevenhua0320

libcblas.dll from conda-forge doesn't work, and libgslcblas-0.dll from mingw64 suggested by ChatGPT also doesn't work. I'm not sure why, but I tried to use the file downloaded from mingw64, renaming it to libcblas.dll, and it worked.

Install mingw64, run pacman -Syu, pacman -S mingw-w64-x86_64-gsl
Copy libgslcbls-0.dll into diffpy.pdffit2/src/diffpy/pdffit2
Rename libgslcbls-0.dll to libcbls.dll.
Modify setup.py, add libgslcbls.dll as package_data

setup_args = dict(
    ext_modules=[],
    cmdclass={"build_ext": CustomBuildExt},
    package_data={"diffpy.pdffit2": ["libcblas.dll"]},  # added
)

Run python -m build

pip install the whl in test-pdffit
run python -c "import diffpy.pdffit2; print(diffpy.pdffit2.__version__)"

I think it is working now. @stevenhua0320, do you need the whl file I built in my local machine?

@ycexiao

Thanks @ycexiao ! could you create a PR for the edits you created and also send the wheel file to both me and @sbillinge so that we can test it out?

I think the edits I made only make sense for windows. Should these be included in our source code?

@ycexiao

@ycexiao Let's ask Simon since it is only pertinent to windows platform. @sbillinge Should we add the libgslcbls.dll and the edits on pyproject.toml directly and make a PR or we add a INSTRUCTION.txt to the users? The former approach does less for the users but these edits would only affect windows users. The latter would make the code structure prettier but a little work for the windows user. What do you think?

@ycexiao

Thanks so much @ycexiao this is amazing! We will have to include that file for it to work on windows, but it is a bit brittle. For example, we are not sure sure if that will work on all windows environments and if there are any updates in the future we would have to keep updating it.

So, tl;dr, lets

@ycexiao make the PR
explore whether we can do some kind of conditional thing in the build where this file is only included in the wheel if it is a windows build.

Sorry just realize this happened. Before we do anything I think it’s better to make the situation clear, because this is not an isolated case. Everything need to be considered with the fact that the same fixing process should also works for diffpy.srreal, libdiffpy, pyobjcryst etc.

The main problem is how to build wheels for PyPI when things are built in a conda-oriented way. Several methods were tried, with the cibuildwheel path and manual build path. I will introduce them one by one.

cibuildwheel path

This path uses the official pypa/cibuildwheel GitHub Action to build wheels on Linux, macOS and Windows in isolated environments.

Good

Standard way to build PyPI wheels:
Cross-platform support out of the box.
Happens in containers, reproducible and isolated builds. it doesn’t pick up arbitrary things from the CI machine or a developer’s conda env, which helps avoid accidentally depending on libraries that users don’t have.

Bad

cibuildwheel runs in isolated build images and does not care about the conda environment. That means:
- Any dependency that exists only in conda is invisible during the build. Even if there happens to be replacement from brew/yum/whatever it needs to be specifically configured in the workflow, which is very hard to make it work.
- setup.py using $CONDA_PREFIX logics will fail.
Depends on the designed dynamic/static linking, setup.py might not behave good enough. For example here it only copies the gsl DLLs, not the blas / cblas DLLs that gsl depends on (e.g. the libcblas stuff from conda-forge).

In a simple way, because of the conda-oriented design, our wheels are very hard to be self-contained in a plain venv, and could very likely works by accident.

Manual conda-based build path

The manual workflow creates a conda environment on ubuntu-latest, installs everything from conda-forge, builds a wheel, and then repairs it using auditwheel plus custom patchelf steps.

Good

The build sees exactly the same conda environment we use for development and conda packaging, which could be self-contained.
All setup.py works as it is, could even modified to work for pure c libraries like libdiffpy.
We don’t need to fight cibuildwheel’s isolation.

Bad

Not thoroughly tested, could break in unexpected ways.
The patchelf logic is non-standard and could be fragile.
Anyone touching the release process needs to understand the surgery well enough.

As for the problem here, the pypi release for this version is using cibuildwheel. Luckily gsl is available outside of conda and the build process happens to work ok. But as said above, it leads to the self-contained problem. The only working states are (which's closely related to how it was built):

in venv
- system level installed gsl on Mac and linux
Or in a conda environment for any platform

If we really want to have working wheels outside of conda, we need to decide which way to go, or maybe other better ways. It's very bad to repeat this every time we make a new release, which inevitably fail if anyone in the chain change anything.

Navigation Menu

Repository navigation

Missing cblas.dll in the Windows pypi wheel #133

Problem

Proposed solution

✅ How Python wheels handle DLLs on Windows

✅ How to include `libcblas.dll` in your wheel

Step 1 — Get `libcblas.dll` from conda-forge

✅ Step 2 — Add the DLL to your wheel using `package_data`

✅ Step 3 — Verify that the wheel contains the DLL

✅ Step 4 — Ensure Python can load the DLL

⭐ Extra Notes

1. Bundling dependencies of dependencies

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Missing cblas.dll in the Windows pypi wheel #133

Description

Problem

Proposed solution

Activity

sbillinge commented on Apr 28, 2025

Tieqiong commented on May 23, 2025

stevenhua0320 commented on Nov 22, 2025

sbillinge commented on Nov 22, 2025

sbillinge commented on Nov 22, 2025

stevenhua0320 commented on Nov 22, 2025

sbillinge commented on Nov 22, 2025

stevenhua0320 commented on Nov 22, 2025

sbillinge commented on Nov 23, 2025

✅ How Python wheels handle DLLs on Windows

✅ How to include libcblas.dll in your wheel

Step 1 — Get libcblas.dll from conda-forge

✅ Step 2 — Add the DLL to your wheel using package_data

✅ Step 3 — Verify that the wheel contains the DLL

✅ Step 4 — Ensure Python can load the DLL

⭐ Extra Notes

1. Bundling dependencies of dependencies

stevenhua0320 commented on Nov 23, 2025

sbillinge commented on Nov 23, 2025

ycexiao commented on Nov 23, 2025

stevenhua0320 commented on Nov 23, 2025

ycexiao commented on Nov 23, 2025

stevenhua0320 commented on Nov 23, 2025

sbillinge commented on Nov 23, 2025

Tieqiong commented on Nov 24, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

✅ How to include `libcblas.dll` in your wheel

Step 1 — Get `libcblas.dll` from conda-forge

✅ Step 2 — Add the DLL to your wheel using `package_data`