Non Portable binary, doesn't work with sudo or without LD_LIBRARY_PATH

Description:
Python binary is compiled with rpath that's /opt/hostedtoolcache/Python. Now, latest github runners define RUNNER_TOOL_CACHE/AGENT_TOOLSDIRECTORY differently than /opt/hostedtoolcache and that installs python at /home/runner/_work/_tool/Python/. So, with this, there are 2 issues.

sudo python --version doesn't work ( See Error section) because most systems's doesn't allow passing LD_LIBRARY_PATH due to security issues.
python --version doesn't work without setting environment variable LD_LIBRARY_PATH

output of ldd :

runner@arss-runner-xxxx-runner-5jvhz:~/_work/_tool/Python/3.12.3/x64/bin$ ldd python3
	linux-vdso.so.1 (0x00007ffceb776000)
	libpython3.12.so.1.0 => not found
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7ccc75c000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f7ccc991000)

Error:

python: error while loading shared libraries: libpython3.9.so.1.0: 
cannot open shared object file: No such file or directory

What could be the reason:

rpath is hardcoded to /opt/hostedtoolcache/Python. if we use $ORIGIN, binaries will be portable. Fix is raised here actions/python-versions#275

Action version:
v5.1.0

Platform:

Ubuntu
macOS
Windows

Runner type:

Hosted
Self-hosted

Tools version:
All versions of Python 3.9.x, 3.10.x, etc.

Repro steps:
You can easily reproduce using following steps. And also, you can unset LD_LIBRARY_PATH and just use without sudo

    steps:
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: |
          sudo python --version

Expected behavior:
There are times we need to use sudo python and Most system ( also not acceptable to pass) doesn't pass LD_LIBRARY_PATH as environment variable due to security issues.

Actual behavior:
sudo python --version to work.

@jaswanthikolla

Hello @jaswanthikolla, Thank you for creating this issue and we will get back to you once we have some feedback on this :)

@aparnajyothi-y I raised the issue just for documentation. PR is already raised to fix this issue, Can you review that.

@jaswanthikolla

@jaswanthikolla thank you for your efforts thus far, it's a shame there hasn't been more movement on this.

@aparnajyothi-y This is affecting certain ansible modules as well now. I am unable to run my checks now due to this.

@priya-kinthali Any update on the PR as it's been 2 months? I see reduced activity with so many open PRs, also see that It's outsourced to some other company. Is this repo still maintained, if not better to handover to other companies which can manage it.

@jaswanthikolla

Hello @jaswanthikolla 👋,
Thank you for providing a detailed description of your issue, and we apologize for the delayed response.
For GitHub runners, the default path is /opt/hostedtoolcache, whereas for self-hosted runners, the runner downloads and installs tools into the folder set up by the RUNNER_TOOL_CACHE environment variable. On Linux self-hosted runners, you can change this location by setting the AGENT_TOOLSDIRECTORY environment variable. This allows you to specify any custom directory to install Python on your self-hosted runner using this variable. For more details, refer to the advanced usage documentation.

Here is a potential workaround that you might consider.

- name: Create Python temp_script.sh  and check python version
  run: |
    PYTHON_PATH=$(which python3)
    PYTHON_DIR=$(dirname $PYTHON_PATH)
    echo "export LD_LIBRARY_PATH=$PYTHON_DIR/../lib" >> temp_script.sh
    echo "export PATH=$PYTHON_DIR:\$PATH" >> temp_script.sh
    echo "python --version" >> temp_script.sh
    chmod +x temp_script.sh
    sudo ./temp_script.sh

Thank you again for the PR. We have already started reviewing it, and testing is underway to check its impact on the overall build environment for both GitHub and self-hosted runners. We will provide feedback once we have tested all possible scenarios.

@jaswanthikolla

Hi @jaswanthikolla ,
side question re

Now, latest github runners define RUNNER_TOOL_CACHE/AGENT_TOOLSDIRECTORY differently than /opt/hostedtoolcache and that installs python at /home/runner/_work/_tool/Python/

On which runners do you see such behaviour? I would expect all github hosted runners to have tool cache under /opt/hostedtoolcache

@jaswanthi

Hello @jaswanthi,
Just a gentle ping to follow up on the previous conversations. Have you had a chance to try the suggested workaround?

On which runners do you see such behaviour? I would expect all github hosted runners to have tool cache under /opt/hostedtoolcache

Additionally could you please confirm?
Thankyou!

@maxim-lobanov

@maxim-lobanov For ARSS runners, the default path is /home/runner/_work/_tool/Python/ .

Yes, that workaround will work, but we pursued another workaround internally. Following up if there are any updates on the PR as It's been 3 months. I can't maintain context on it forever to address review comments and so I guess it's time to abandon/close the PR.

I'm not exactly sure why there is a lengthy discussion around this issue. Using ORIGIN is typical in the *nix world, and using hardcoded path (except maybe the standard one like /usr/local/lib) is usually seen as an anti-pattern. In the current case, using a hardcoded path prevents actions/setup-python from being used easily on self-hosted GitHub runners (our case).

Of course there are workarounds, but they decrease the quality of user experience and they increase the maintenance burden of users for their actions and workflows. So let's reverse the question: what are the downsides of starting to use ORIGIN as already implemented in the pull request when compared to the status quo? 🤔

If there is no significant downsides, why not just benefit from the community's work and merge the pull request ASAP? 😎

@klalumiere

@klalumiere I've resorted to forking both repositories and using that for now. It's been months without a resolution despite @jaswanthikolla's efforts. Her PR works and should be merged.

Here is a potential workaround that you might consider.
- name: Create Python temp_script.sh  and check python version
  run: |
    PYTHON_PATH=$(which python3)
    PYTHON_DIR=$(dirname $PYTHON_PATH)
    echo "export LD_LIBRARY_PATH=$PYTHON_DIR/../lib" >> temp_script.sh
    echo "export PATH=$PYTHON_DIR:\$PATH" >> temp_script.sh
    echo "python --version" >> temp_script.sh
    chmod +x temp_script.sh
    sudo ./temp_script.sh       
Thank you again for the PR. We have already started reviewing it, and testing is underway to check its impact on the overall build environment for both GitHub and self-hosted runners. We will provide feedback once we have tested all possible scenarios.

This workaround does not work on self-hosted runners with pantsbuild.

We're also affected by this (with our self-hosted runners), did not realize by the end of 2024 there are still real-world Python compiled without relocatability 😢 This is very frustrating for people trying to rely more on GHA and less on custom Python distros (ex: conda)...

Abandon setup-python

Hi

I'd like to raise another problem with the LD_LIBRARY_PATH approach — this overrides RUNPATH which means that you can break other Python interpreters on the system. For example, I'm working on musl distributions of CPython over in astral-sh/python-build-standalone#541 and our tests failed in CI because LD_LIBRARY_PATH is set and our Python interpreter loaded the GitHub Actions libpython instead of ours. A bit of an side, but we can't use DT_NEEDED (which takes precedence over LD_LIBRARY_PATH) instead of RUNPATH there because musl doesn't support the $ORIGIN syntax in it.

Using $ORIGIN really is best practice here. If there are downsides to that approach, I'd love to hear it. We've been using $ORIGIN for relocatable Python distributions in python-build-standalone for years now without issue.

@priya-kinthali please merge this

fix(k9b-live-lab-toolchain): wire LD_LIBRARY_PATH to fix libpython3.1…

Navigation Menu

Repository navigation

Non Portable binary, doesn't work with sudo or without LD_LIBRARY_PATH #871

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Non Portable binary, doesn't work with sudo or without LD_LIBRARY_PATH #871

Description

Activity

v-aparnajyothi-y commented on May 20, 2024

jaswanthikolla commented on May 20, 2024

navarro967 commented on May 31, 2024

jaswanthikolla commented on Jul 19, 2024

v-priya-kinthali commented on Jul 19, 2024

maxim-lobanov commented on Jul 23, 2024

v-priya-kinthali commented on Aug 7, 2024

jaswanthikolla commented on Aug 14, 2024

klalumiere commented on Aug 23, 2024

navarro967 commented on Aug 23, 2024

WTPOptAxe commented on Nov 7, 2024

leofang commented on Dec 5, 2024

zanieb commented on Feb 27, 2025

mike-essential commented on Mar 17, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions