Repository navigation
Py_GetProgramFullPath() odd behaviour in Windows #78906
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2018-09-18.17:26:47.540>
labels = ['interpreter-core', '3.7', '3.8', 'OS-windows']
title = 'Py_GetProgramFullPath() odd behaviour in Windows'
updated_at = <Date 2019-05-27.21:30:46.561>
user = 'https://bugs.python.org/mariofutire'bugs.python.org fields:
activity = <Date 2019-05-27.21:30:46.561>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core', 'Windows']
creation = <Date 2018-09-18.17:26:47.540>
creator = 'mariofutire'
dependencies = []
files = ['47814']
hgrepos = []
issue_num = 34725
keywords = ['patch']
message_count = 23.0
messages = ['325666', '325668', '325669', '325674', '326035', '326042', '326970', '327000', '327081', '327249', '327364', '327370', '327447', '327489', '327659', '327701', '328054', '330036', '330037', '330038', '343644', '343681', '343691']
nosy_count = 8.0
nosy_names = ['paul.moore', 'ncoghlan', 'vstinner', 'tim.golden', 'eric.snow', 'zach.ware', 'steve.dower', 'mariofutire']
pr_nums = ['9860', '9861']
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue34725'
versions = ['Python 3.7', 'Python 3.8']
Activity
zooba commentedon Oct 10, 2018
We'll need to bring in venv specialists to check whether using it outside of Py_Main() is valid. Or perhaps you could explain what you are actually trying to do?
I don't believe it is necessary when you are calling Py_SetPath yourself, and only the "launch normally with alternate args" case for scripts that use sys.executable are affected. But I'm happy to be set right here (with example scenarios, preferably).
mariofutire commentedon Oct 10, 2018
On 10/10/2018 01:11, Steve Dower wrote:
Sure
Result
Works in Linux, fails in Windows.
Reason in site.py
cpython/Lib/site.py
Line 462 in 73870bf
sys.executable is used to construct the correct search path.
Looking at the sys.path from inside an embedded application is very instructive and you can see in
the first post why the failure in windows.
Andrea
zooba commentedon Oct 13, 2018
I meant why are you using an embedded application with a virtual environment? What sort of application do you have that requires users to configure a virtual environment, rather than providing its own set of libraries?
The embedding scenarios I'm aware of almost always want privacy/isolation from whatever a user has installed/configured, so that they can work reliably even when users modify other parts of their own system. I'm trying to understand what scenario (other than "I am an interactive Python shell") would want to automatically pick up the configuration rather than having its own configuration files/settings.
mariofutire commentedon Oct 14, 2018
On 13/10/2018 17:37, Steve Dower wrote:
Does it really matter who owns main(), whether it is in python.exe or in some other C app.
This is exactly how you described, users want to use some C application which will call into python
using some (user defined) python modules to execute some tasks which are scriptable.
And they want to be able to do in a confined environment where they can install the exact set of
packages they require. And it is possible at the same time to set up multiple environments where
different versions are tested independently.
There is as well the totally independent scenario where the app ships exactly what it needs, but
there are some ways in between where one can script an app and in doing so you might need packages
that the app itself knew nothing about.
For another example have a look at JEP
https://github.com/ninia/jep/search?q=virtual&unscoped_q=virtual
This is a way to call python from Java: same problem above, people might want to run it in a virtual
environment and the only way to do this now is to manually set up PYTHONHOME, but it is pretty weak
and does not replicate exactly what happens with virtual environments (e.g. inherit system's
site-packages).
Again, in Linux, JEP works out of the box with no need to tell it about virtual environments,
Py_Initialise() finds it (if they are indeed present) with absolutely no extra configuration (no
need to change PYTHONPATH).
Andrea
zooba commentedon Oct 19, 2018
I requested Victor review on my PR, but if anyone else is able to please feel free.
zooba commentedon Nov 18, 2018
New changeset 177a41a by Steve Dower in branch 'master':
bpo-34725: Adds _Py_SetProgramFullPath so embedders may override sys.executable (GH-9860)
177a41a
zooba commentedon Nov 18, 2018
New changeset e851049 by Steve Dower in branch '3.7':
bpo-34725: Adds _Py_SetProgramFullPath so embedders may override sys.executable (GH-9861)
e851049
zooba commentedon Nov 18, 2018
The next releases of 3.7 and 3.8 will include _Py_SetProgramFullPath() functions for embedders to set the eventual value of sys.executable before calling Py_Initialize(). It's undocumented and not guaranteed stable (and indeed, it looks like Victor is already working on another patch that may see it removed before it's ever released), but it's in now as a workaround for the cases that need it.
vstinner commentedon May 27, 2019
I understand that issue is now fixed in bpo-36763 by the implementation of the PEP-587 which adds a new public API for the "Python Initialization Configuration". It provides a finer API to configure the "Path Configuration". For example, PyConfig.executable can be used to replace _Py_SetProgramFullPath().
In Python 3.7, the private _Py_SetProgramFullPath() function can be used as a workaround.
I close the issue. If I misunderstood the issue, please comment/reopen it ;-)
mariofutire commentedon May 27, 2019
Unfortunately the underlying cause of this issue has not been addressed, nor discussed.
There is now a way to workaround the different behaviour in Windows and Linux and it is possible to use the new call to make virtual environment work in Windows as they already do in Linux.
Problem is that application will have to be change to actually implement the workaround.
I still think this difference should be addressed directly.
vstinner commentedon May 27, 2019
I read again the issue. In short, the Path Configuration is a mess and unusable in some cases :-) I reopen the issue.
Handling venv shouldn't be handled by the site module which is optional, but earlier.
I guess that venv support was added to site because it is way eaiser to write Python, than touching getpath.c written in C.
FFY00 commentedon Mar 3, 2026
That changed in GH-126987 — since 3.14, the venv support is now part of getpath.
I think the reported difference in
sys.pathon Windows should now be resolved, but it would be good to have someone confirm this on a Windows machine.Regarding the executable path calculation,
sys.executableshould be the same for all platforms.sys._base_executablestill has the discussed special case on Windows, though this shouldn't affectsys.path.