Skip to content

Fixes and workarounds for Conda build - #24

Closed
roryyorke wants to merge 1 commit into
python-control:masterfrom
roryyorke:rory/build-fix
Closed

roryyorke wants to merge 1 commit into
python-control:masterfrom
roryyorke:rory/build-fix

Conversation

@roryyorke

Copy link
Copy Markdown
Collaborator

Still in progress, but I have successfully built on Travis with Conda with Python 2.7, 3.5, and 3.6. The CI build fails with Python 3.6 in the non-Conda build; I'm still investigating that. See https://travis-ci.org/roryyorke/Slycot/builds/391434012

The main changes:

  • openblas instead of lapack (openblas supplies liblapack and libblas)
  • add -shared to LDFLAGS instead of overwriting it
  • use new meta.yaml layout (not sure if this is critical)

To-do; would appreciate any assistance with these:

  • test on OSX (I can't do this)
  • test on Windows, in particular check I haven't broken @repagh's work on Windows builds
  • investigate test failure

meta.yaml:
  - update to newer conda file layout
  - depend on openblas instead of lapack

build.sh:
  - correct LDFLAGS setup
  - workaround how Conda setups up Fortran compilers
@murrayrm

murrayrm commented Jun 18, 2018 •

Copy link
Copy Markdown
Member

I tried getting conda build to work on MacOS 10.13.5 with python 3.6 but had problems. When I do

conda build conda-recipes

the entire build process is carried out and it looks like everything is fine up until the final test. At the very end I get the error

import: 'slycot'
Fatal Python error: PyThreadState_Get: no current thread

This error is occurring on the final step of the build process, where conda attempts to import slycot. Near as I can tell from looking online this is a problem with my local python installation, but I have not yet been able to sort it out.

@murrayrm

Copy link
Copy Markdown
Member

Quick update: I am having the same problem doing a "standard" build: if I use python setup.py build followed by python runtests.py then I get the same error. So not directly related to conda?

@roryyorke

Copy link
Copy Markdown
Collaborator Author

I know nothing about OS X, but from snap-stanford/snap#88 (comment) it looks like this is a Slycot build error.

I did change the build flags in build.sh (LDFLAGS and FFLAGS) - perhaps one of those is the culprit?

Separately, I'm still struggling to get Slycot to build on Windows.

@repagh

repagh commented Jun 23, 2018 •

Copy link
Copy Markdown
Member

I found the import slycot error with the PyThreadState_Get issue.
We use -lpython... to link the unknown symbols (OSX by default does not allow undefined symbols for a dynamic library, which a python module essentially is). This works if Python actually uses the libpython dylib. New conda has a python executable that has all basic python symbols statically linked in (if you are going to use them anyway, why not....). To fix the build (for now on repagh/slycot, master) I removed the library dependency, and added
-Wl,-dylib,-undefined,dynamic_lookup linker flags. That solved the load problem.

Use of openblas is interesting, where do you get it from, and is it built for all three platforms? I know I tried about a year ago, but could not get the conda openblas for windows. Maybe things are better now, since we can use compilers+linkers from conda.

By the way, I now have that slycot version + control for all three platforms on my anaconda channel (repa).

Greetings,
René

@roryyorke

roryyorke commented Jun 24, 2018 via email

Copy link
Copy Markdown
Collaborator Author

@repagh

repagh commented Jun 26, 2018

Copy link
Copy Markdown
Member

On the repagh/Slycot master branch (I know, I shouldn't), I got the Windows build working. Still need to figure out how to nicely git-push that on your work.

Solution chosen:

  • compilation of the Fortran code with Flang. To get this to work, needs defines for F77 and F90 in the build script, otherwise numpy's setup script fails, since that does not "know" Flang yet. Flang can be installed through conda, and is a dependency in meta.yaml
  • linking and compilation of c parts with visual studio community edition 2015. This needs a separate install of the community edition (I put the download link in the meta.yaml, without that, MicroSoft tries to push the latest and greatest on you). The associated conda package sets all environment variables correctly.
  • building against openblas from conda-forge. Had to make some small tweaks in slycot/setup.py

The current conda packages are in the "repa" conda channel.

@repagh

repagh commented Jun 27, 2018

Copy link
Copy Markdown
Member

Rory, I think your CONDA=0 builds use lapack instead of openblas, at least that is what you install, and link to in the slycot/setup.py

@murrayrm

Copy link
Copy Markdown
Member

It sounds like we are going to have a problem supporting different installation environments. One tool that we might use is python-config --ldflags, which can be used to ask the local python installation what flags to use. More info here: https://docs.python.org/3/extending/embedding.html#compiling-and-linking-under-unix-like-systems.

@murrayrm

Copy link
Copy Markdown
Member

This PR is superseded by PR #26.

@murrayrm murrayrm closed this Jun 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants