Skip to content

Pyston compatibility for the test suite - #7901

Merged
sklam merged 2 commits into
numba:mainfrom
kmod:pyston_compat
Mar 14, 2022
Merged

sklam merged 2 commits into
numba:mainfrom
kmod:pyston_compat

Conversation

@kmod

@kmod kmod commented Mar 10, 2022

Copy link
Copy Markdown
Contributor

numba already works on Pyston, but there were two small snags in the test suite that caused the tests to not pass.

@gmarkall

Copy link
Copy Markdown
Member

This is exciting, I didn't even know Numba worked on Pyston! Slightly tangential, but how easy is it to add Pyston to the Azure CI tests?

@stuartarchibald

Copy link
Copy Markdown
Contributor

This is exciting, I didn't even know Numba worked on Pyston! Slightly tangential, but how easy is it to add Pyston to the Azure CI tests?

Indeed, great to see Numba works with Pyston.

RE Azure. I think it'd require a llvmlite development conda package that works under Pyston adding to the numba channel somewhere. Other than that I think it's just selectively adding in some conda channels and probably a e.g. $PYTHON_IMPL identifier to the build scripts/Azure config.

@stuartarchibald stuartarchibald added this to the Numba 0.56 RC milestone Mar 11, 2022

@stuartarchibald stuartarchibald left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this manually on Pyston with/without this patch. This patch fixes the noted failing tests. Many thanks.

@stuartarchibald stuartarchibald added 5 - Ready to merge Review and testing done, is ready to merge and removed 3 - Ready for Review labels Mar 11, 2022
@stuartarchibald

Copy link
Copy Markdown
Contributor

Notes:

  • Initially CUDA tests were failing on Pyston, segfault very early on in library loading, I then installed SciPy from conda-forge and it seemed to "fix it" and all the CUDA tests passed, even though the SciPy build itself was broken, no idea why as yet (imagine it's to do with changing some underlying libraries to get SciPy to install)!
  • Seems like there's no suitable SciPy build for Pyston so could not run the numba.tests.test_linalg tests which use SciPy's BLAS/LAPACK bindings.

@seibert

seibert commented Mar 11, 2022

Copy link
Copy Markdown
Contributor

One note (mentioned OOB, but putting here for posterity) is that current Pyston conda packages are built in the conda-forge style, using the python_abi method for managing alternative interpreters. Anaconda's packages don't yet do this, so testing with Pyston should be done in an environment built this way to avoid contamination by packages with an incompatible ABI:

conda create -n pyston_env  # create an empty environment
conda activate pyston_env
echo 'channels: [pyston,conda-forge]' | conda config --env --stdin
conda install pyston # add whatever other packages you want

(reference: https://github.com/pyston/pyston/wiki/Using-conda)

@sklam
sklam merged commit 2e825b0 into numba:main Mar 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

5 - Ready to merge Review and testing done, is ready to merge Effort - short Short size effort needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants