Repository navigation
User site packages are being added when site module is enabled in _pth file #120037
Description
Activity
This is a straight-forward fix. Add the statement
config['user_site_directory'] = 0in the following section:Lines 760 to 764 in 69b3e8e
Reacted by Steve Dower and TanzimWhile reproducing this, I noted something else on
macOS- this is assuming Python 3.11 adds support for._pthfile on macOS:- Placing
._pthfile with interpreter executable name in the interpreter dir, e.g.binworks - Enabling site has the same issue as above
- However, no matter how much I tried I couldn't get a
.pthfile to work, doesn't matter where I placed it.
- Placing
- changed the title
[-][Windows] User site packages are being added when site module is enabled in _pth file[/-][+]User site packages are being added when site module is enabled in _pth file[/+]on Jun 5, 2024 However, no matter how much I tried I couldn't get a
.pthfile to work, doesn't matter where I placed it.On Windows, the
sitemodule executes.pthfiles found in the prefix directory; the site-packages directory "{prefix}\Lib\site-packages"; and, if enabled, the user site-packages directory. On POSIX, it checks the site-packages directory and, if enabled, the user site-packages directory, but not the prefix directory. The path of the site-packages directory on POSIX is usually of the form "{prefix}/lib/pythonX.Y/site-packages", unless a distribution customizes it.After adding the site-packages and user site-packages directories, and executing their
.pthfiles, thesitemodule tries to import asitecustomizemodule. This may be simpler for you to manage since it doesn't matter wheresitecustomizeis located.On POSIX, it checks the site-packages directory and, if enabled, the user site-packages directory, but not the prefix directory.
That explains why the
.pthfile didn't work for me same way as Windows even though I tried placing it in prefix, prefix/bin and site packages as above. I can confirmsitecustomizeworks. I have to say though, the various initialization logic is pretty convoluted!Thanks again for the help.
Could be another esoteric piece of information that I lack however I noted that when using
sitecustomizeon macOS, even though import site in defined in the._pthfile, site-packages directory, e.g./lib/python3.11/site-packagesis not added tosys.pathby default. On Windows it is. I could of course manually add it in the._pthfile but that feels hacky.Could be another esoteric piece of information that I lack however I noted that when using
sitecustomizeon macOS, even though import site in defined in the._pthfile, site-packages directory, e.g./lib/python3.11/site-packagesis not added tosys.pathby default. On Windows it is. I could of course manually add it in the._pthfile but that feels hacky.Maybe the value of
sys.prefixis wrong. I don't use macOS, so I can't test this.Reacted by TanzimYou guessed right. It's pointing to python/bin/lib/python3.11/site-packages and sys.prefix is set to python/bin when it should be one level up.
I'm pretty sure this is a side effect of colocating the
._pthfile with the python executable in the bin folder. If I remove the_.pthsys.prefixworks as expected andsite.getsitepackages()returns the correct value.I'm pretty sure this is a side effect of colocating the
._pthfile with the python executable in the bin folder.If a
._pthfile is found, its parent directory gets set ashomein "Modules/getpath.py". This setssys.prefix1. If you need it to be a level up, can you run from a "python" symlink that's created a level up? A._pthfile can be located beside a symlink to the executable or beside the real executable.Footnotes
-
Except when running from the build directory on POSIX, the default prefix is used. ↩
-
Yeah, having
._pthdirectly dictate the prefix isn't ideal, but it's also a difficult one to generalise the rules for.Fundamentally, the idea is that your
._pthfile should prevent/limit the Python runtime from being exploitable through path hijacks, but as soon as we start adding rules like "Sometimes sys.prefix is the directory above the file" or deriving it from user-provided/environment data like normal, we break the security rule.I think the right fix here is to add the
site-packagesdirectory to the._pthfile if you want it. The main scenario is for embedding, when you probably aren't using (and certainly don't need) thesite-packagesconvention at all. Still, it's a bit of a shame that it is deriving fromsys.prefixrather than the stdlib path (maybe we don't even calculate this though...?).7 remaining items
- Getting isolated mode, safe-path mode, and (once fixed) disabling the user site-packages, is contingent on the
._pthfile being non-empty, in which case its contents determine the initialsys.path.
Yeah, this is the intended use. An empty
._pthought to just fail - not entirely obvious why it doesn't right now - because it should result in an emptysys.path. I guess there are some additional checks in getpath.py that look for empty lists...Because https://github.com/python/cpython/blob/main/Modules/getpath.py#L669 and https://github.com/python/cpython/blob/main/Modules/getpath.py#L723
It's baked in Windows behavior.
- Getting isolated mode, safe-path mode, and (once fixed) disabling the user site-packages, is contingent on the
Yeah, this is the intended use. An empty
._pthought to just fail - not entirely obvious why it doesn't right now - because it should result in an emptysys.path. I guess there are some additional checks in getpath.py that look for empty lists...An empty
._pthfile setsprefix, but it doesn't preclude the normal code that sets up the initial module search path inpythonpath, except to skip anything from the environment/registry becauseuse_environmentis 01. The way it's designed is that if thepthvariable is non-empty from the._pthfile, then its contents replaces the calculated value ofpythonpath, in addition to settingconfigvalues that isolate the application.Footnotes
-
But not
config['use_environment'], so other Python environment variables still affect the application. ↩
-
By "intended" I meant what I intended when I originally added it, not when it got translated into
getpath.py😄But it does seem reasonable to allow it to generate a relatively default path if it's empty. It's very easy to avoid that by just not making it empty, which is going to be necessary most of the time anyway.
Bumping this year-old issue, I have an interest in seeing this issue resolved 😅
It looks like the CLA is the only thing actually blocking the PR right now?
I have an interest in seeing this issue resolved
I assume you mean the original issue, and not any of the others discussed in the thread.
Could you help me understand why you need to use
import sitein your._pthfile? The intent of the._pthis to avoid arbitrary paths being added in, and while there is an escape hatch available, it would be better to not need it.I assume you mean the original issue, and not any of the others discussed in the thread.
Yes, only the original issue.
Could you help me understand why you need to use import site in your ._pth file? The intent of the ._pth is to avoid arbitrary paths being added in, and while there is an escape hatch available, it would be better to not need it.
I've had some more time to think of how to deal with my situation since 4 days ago, and I know there are ways to get what I want without
import sitein the._pthfile, so I'm not going to say I need this ;) But it would make what I'm trying to do a fair bit easier.The gist of it is that I'm upgrading a C++ application that embeds Python from Python 2 to 3 (finally!!!). I'm taking the opportunity to remove our dependency on a system-wide Python installation, by shipping the embeddable python package with a
._pthfile. We also have several Python packages that need to be included and available to the embedded Python interpreter, but those packages cannot be baked into the embeddable package, for organisational reasons basically.I can fairly easily add those packages from the C++ application with
site.addsitedir()right after the Python initialization. (Some of the packages have .pth files associated, e.g. pywin32, so just adding the path tosys.pathfrom the._pthfile is insufficient.) But there are also situations where we need to invokepython.exefrom outside the application, and it's more convenient if those packages are immediately available in that context as well. The first solution I reached for wasimport sitein the._pthfile with asitecustomize.pyif needed, but this bug causes user-site packages to be added as well, thus de-isolating the environment with unpredictable side-effects.Thanks, that's useful context, and it's definitely the kind of thing we want to enable.
We have a PR that would fix it, I think it just got held up on improving the description of the issue?
We have a PR that would fix it, I think it just got held up on improving the description of the issue?
Seems like it's that + accepting the Contributor's License Agreement. Though it looks like the author @tanzim hasn't shown any recent activity. Would it be appropriate for me to open a new PR then?
- added a commit that references this issue
on Aug 19, 2025 No, I think it's too big a behaviour change, and not justified as a security fix (if you can write to the user profile you're already far enough in - no escalation of privilege here). Embedding apps that use
python.exeare already more open to misuse than this change would allow, and those that load the DLL directly can probably set the flag themselves more easily than updating the runtime. It'll be in 3.15.Reacted by Martin BoisvertUndersood. Thanks again!
Bug report
Bug description:
See #119920 (comment).
I've managed to reproduce this with the latest official Windows embedded Python releases for 3.11 and 3.12.
CPython versions tested on:
3.11, 3.12
Operating systems tested on:
Windows
Linked PRs