Repository navigation
"Building Linux wheels for Python 3.8 requires a compiler (e.g gcc)." #123
Description
Activity
It's also worth mentioning that gcc is in fact installed in the container:
$ gcc gcc: fatal error: no input files compilation terminated.Hi,
To quote from the version 0.3.5 README:
If a pre-complied binary (i.e. "wheel") is available it will be installed with no need for compilation. If not, pip will attempt to compile the package from source and thus the compilation dependencies will be required (scikit-build, gfortran, BLAS, etc.).
From your log:
-- The C compiler identification is GNU 8.3.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc - works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- The Fortran compiler identification is unknown CMake Error at CMakeLists.txt:4 (ENABLE_LANGUAGE): No CMAKE_Fortran_COMPILER could be found.gcc 8.3.0 is found, but gfortran is not installed. The generic scikit-build message does not differentiate between the C and Fortran compiler.
Yeah, as I discovered the magic incantation is
sudo apt install gfortran libblas-dev liblapack-dev... or any other more performant blas/lapack implementation
I'm running into the same problem in a CI build job using Ubuntu 18.04 with ROS in a docker container, which installs slycot automatically via pip (as a dependency of our application code).
As the slycot pip build depends on gfortran, but doesn't provide it as dependency itself, the build fails.
Even, if I manually specify gfortran as additional dependency of my application, the build does not always succeed, because we can't define the order in which the dependencies are installed (rosdep only follows the dependency tree).I'm not an expert on pip, but would it be possible to either
- provide pip wheels (at least for Ubuntu LTS versions) to prevent building from source or
- define gfortran as a dependency of slycot, install it via build scripts, … (as said, I'm not really aware of how pip works with this regards)?
I'm not an expert on pip, but would it be possible to either
- provide pip wheels (at least for Ubuntu LTS versions) to prevent building from source or
Wheels are not distro specific. When we provide wheels, which are linked with the blas/lapack libraries of one Ubuntu LTS, people will try to install them on different Linux distros and have a horrible user experience. The NumPy and SciPy wheels provide their own lapack/libraries and we would have to sync with them in order to avoid trouble.
- define gfortran as a dependency of slycot, install it via build scripts, … (as said, I'm not really aware of how pip works with this regards)?
As far as I am aware, pip only follows the Python package dependencies specified via setuptools. A Fortran compiler is not a Python package. It has to be provided through the operating system before running pip install. For Ubuntu, that is
apt-get install gfortranas mentioned above.I am not familiar with ROS containers. Here are two ideas, which you could try:
- Find a place or hook in your CI/container-creation/ROS-definition which can call
apt-get install gfortran ...before thepip install your-application - Create your own custom wheel in a separate Ubuntu environment with
pip wheeland inject the file into the docker system
If you need pre-compiled packages, consider using the conda ecosystem instead of pip. Or use a distribution which provides distro specific packages. I know of Archlinux (AUR) and openSUSE Tumleweed.
Even, if I manually specify gfortran as additional dependency of my application, the build does not always succeed, because we can't define the order in which the dependencies are installed (rosdep only follows the dependency tree).
Hmm, that sounds more like you have specified a runtime dependency where a buildtime dependency should have been used. Somehow you must specify to rosdep that gfortran is a build requirement for Slycot.
Hmm, that sounds more like you have specified a runtime dependency where a buildtime dependency should have been used. Somehow you must specify to rosdep that gfortran is a build requirement for Slycot.
Actually, I did specify gfortran as a build dependency of my ROS package:
<package format="3"> <name>my_ros_pkg</name> […] <buildtool_depend>gfortran</buildtool_depend> <exec_depend>python-control-pip</exec_depend> <exec_depend>python-slycot-pip</exec_depend> </package>
but rosdep is interpreting
buildtool_dependasbuild system tools which this package needs to build itself
so it thinks gfortran is needed to build
my_ros_pkg, not to build python-slycot-pip.
Which makes sense, because gfortran is a build dependency of slycot – notmy_ros_pkg.That's why I'm asking why pip is not installing gfortran, if it needs it or how to get it to do so?
As a workaround I could try to setup a wrapper package with gfortran as build dependency and then add this wrapper package to
my_ros_pkg.
Still, this feels hacky…That's why I'm asking why pip is not installing gfortran, if it needs it or how to get it to do so?
Because pip can only install python packages. It is not meant to install system packages at all. If you run pip as root it gives you a stern warning.
I guess you have to fix the ROS package
python-slycot-pip.You're right, pip wouldn't be able to install system packages anyways (And using pip as root is a bad idea, because it can break the system Python installation).
Unfortunately there is no ROS package
python-slycot-pipI could fix. It is simply a dependency definition telling rosdep (the ROS dependency manager) to resolve it to thepython-slycotpip package – probably assuming that all pip packages will be installed from wheels, if they contain binaries.
And it seems that I can't add build dependencies to this rosdep entry…Btw. changing the OS is not an option, as ROS is (more or less) tightly coupled to Ubuntu.
Neither is conda supported by rosdep 😖I'll take care to install gfortran in all our docker containers and physical machines then (won't hurt anyways) 💁♂️
Thanks for your explanations and efforts @bnavigator !
Unfortunately there is no ROS package
python-slycot-pipI could fix.You could create a
python-slycotROS package (or even a Debian/Ubuntu one).I'll take care to install gfortran in all our docker containers and physical machines then (won't hurt anyways)
Don't forget cmake and the lapack/blas headers. Have a look at our GitHub Actions workflow for hints:
Slycot/.github/workflows/slycot-build-and-test.yml
Lines 88 to 99 in a40505a
Slycot/.github/workflows/slycot-build-and-test.yml
Lines 115 to 122 in a40505a
Thanks for the hint!
The pip install process for slycot fails on most clean linux installs. Here are the results running in a
circleci/pythonDocker container:But the thing is... running
sudo apt-get install build-essentialdoesn't help at all. Installing build-essential and then re-running results in exactly the same error.I don't have issues pip installing any other packages in the same container.