Repository navigation
Make pyvenv style virtual environments easier to configure when embedding Python #66409
Description
Activity
grahamd commented
on Aug 17, 2014 grahamdmannequinMannequinAuthorIn am embedded system, as the 'python' executable is itself not run and the Python interpreter is initialised in process explicitly using PyInitialize(), in order to find the location of the Python installation, an elaborate sequence of checks is run as implemented in calculate_path() of Modules/getpath.c.
The primary mechanism is usually to search for a 'python' executable on PATH and use that as a starting point. From that it then back tracks up the file system from the bin directory to arrive at what would be the perceived equivalent of PYTHONHOME. The lib/pythonX.Y directory under that for the matching version X.Y of Python being initialised would then be used.
Problems can often occur with the way this search is done though.
For example, if someone is not using the system Python installation but has installed a different version of Python under /usr/local. At run time, the correct Python shared library would be getting loaded from /usr/local/lib, but because the 'python' executable is found from /usr/bin, it uses /usr as sys.prefix instead of /usr/local.
This can cause two distinct problems.
The first is that there is no Python installation at all under /usr corresponding to the Python version which was embedded, with the result of it not being able to import 'site' module and therefore failing.
The second is that there is a Python installation of the same major/minor but potentially a different patch revision, or compiled with different binary API flags or different Unicode character width. The Python interpreter in this case may well be able to start up, but the mismatch in the Python modules or extension modules and the core Python library that was actually linked can cause odd errors or crashes to occur.
Anyway, that is the background.
For an embedded system the way this problem was overcome was for it to use Py_SetPythonHome() to forcibly override what should be used for PYTHONHOME so that the correct installation was found and used at runtime.
Now this would work quite happily even for Python virtual environments constructed using 'virtualenv' allowing the embedded system to be run in that separate virtual environment distinct from the main Python installation it was created from.
Although this works for Python virtual environments created using 'virtualenv', it doesn't work if the virtual environment was created using pyvenv.
One can easily illustrate the problem without even using an embedded system.
$ which python3.4 /Library/Frameworks/Python.framework/Versions/3.4/bin/python3.4 $ pyvenv-3.4 py34-pyvenv $ py34-pyvenv/bin/python Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 00:54:21) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.prefix '/private/tmp/py34-pyvenv' >>> sys.path ['', '/Library/Frameworks/Python.framework/Versions/3.4/lib/python34.zip', '/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4', '/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/plat-darwin', '/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/lib-dynload', '/private/tmp/py34-pyvenv/lib/python3.4/site-packages'] $ PYTHONHOME=/tmp/py34-pyvenv python3.4 Fatal Python error: Py_Initialize: unable to load the file system codec ImportError: No module named 'encodings' Abort trap: 6
The basic problem is that in a pyvenv virtual environment, there is no duplication of stuff in lib/pythonX.Y, with the only thing in there being the site-packages directory.
When you start up the 'python' executable direct from the pyvenv virtual environment, the startup sequence checks know this and consult the pyvenv.cfg to extract the:
home = /Library/Frameworks/Python.framework/Versions/3.4/bin
setting and from that derive where the actual run time files are.
When PYTHONHOME or Py_SetPythonHome() is used, then the getpath.c checks blindly believe that is the authoritative value:
- Step 2. See if the $PYTHONHOME environment variable points to the
- installed location of the Python libraries. If $PYTHONHOME is set, then
- it points to prefix and exec_prefix. $PYTHONHOME can be a single
- directory, which is used for both, or the prefix and exec_prefix
- directories separated by a colon.
/* If PYTHONHOME is set, we believe it unconditionally */ if (home) { wchar_t *delim; wcsncpy(prefix, home, MAXPATHLEN); prefix[MAXPATHLEN] = L'\0'; delim = wcschr(prefix, DELIM); if (delim) *delim = L'\0'; joinpath(prefix, lib_python); joinpath(prefix, LANDMARK); return 1; } Because of this, the problem above occurs as the proper runtime directories for files aren't included in sys.path. The result being that the 'encodings' module cannot even be found.
What I believe should occur is that PYTHONHOME should not be believed unconditionally. Instead there should be a check to see if that directory contains a pyvenv.cfg file and if there is one, realise it is a pyvenv style virtual environment and do the same sort of adjustments which would be made based on looking at what that pyvenv.cfg file contains.
For the record this issue is affecting Apache/mod_wsgi and right now the only workaround I have is to tell people that in addition to setting the configuration setting corresponding to PYTHONHOME, to use configuration settings to have the same effect as doing:
PYTHONPATH=/Library/Frameworks/Python.framework/Versions/3.4/lib/python34.zip:/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4:/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/plat-darwin:/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/lib-dynload
so that the correct runtime files are found.
I am still trying to work out a more permanent workaround I can add to mod_wsgi code itself since can't rely on a fix for existing Python versions with pyvenv support.
Only other option is to tell people not to use pyvenv and use virtualenv instead.
Right now I can offer no actual patch as that getpath.c code is scary enough that not even sure at this point where the check should be incorporated or how.
Only thing I can surmise is that the current check for pyvenv.cfg being before the search for the prefix is meaning that it isn't consulted.
/* Search for an environment configuration file, first in the executable's directory and then in the parent directory. If found, open it for use when searching for prefixes. */ { wchar_t tmpbuffer[MAXPATHLEN+1]; wchar_t *env_cfg = L"pyvenv.cfg"; FILE * env_file = NULL; wcscpy(tmpbuffer, argv0_path);joinpath(tmpbuffer, env_cfg); env_file = _Py_wfopen(tmpbuffer, L"r"); if (env_file == NULL) { errno = 0; reduce(tmpbuffer); reduce(tmpbuffer); joinpath(tmpbuffer, env_cfg); env_file = _Py_wfopen(tmpbuffer, L"r"); if (env_file == NULL) { errno = 0; } } if (env_file != NULL) { /* Look for a 'home' variable and set argv0_path to it, if found */ if (find_env_config_value(env_file, L"home", tmpbuffer)) { wcscpy(argv0_path, tmpbuffer); } fclose(env_file); env_file = NULL; } }
pfound = search_for_prefix(argv0_path, home, _prefix, lib_python);
Yeah, PEP-432 (my proposal to redesign the startup sequence) could just as well be subtitled "getpath.c hurts my brain" :P
One tricky part here is going to be figuring out how to test this - perhaps adding a new test option to _testembed and then running it both inside and outside a venv.
Graham pointed out that setting PYTHONHOME ends up triggering the same control flow through getpath.c as calling Py_SetPythonHome, so this can be tested just with pyvenv and a suitably configured environment.
It may still be a little tricky though, since we normally run the pyvenv tests in isolated mode to avoid spurious failures due to bad environment settings...
Some more experiments, comparing an installed vs uninstalled Python. One failure mode is that setting PYTHONHOME just plain breaks running from a source checkout (setting PYTHONHOME to the checkout directory also fails):
$ ./python -m venv --without-pip /tmp/issue22213-py35 $ /tmp/issue22213-py35/bin/python -c "import sys; print(sys.base_prefix, sys.base_exec_prefix)" /usr/local /usr/local $ PYTHONHOME=/usr/local /tmp/issue22213-py35/bin/python -c "import sys; print(sys.base_prefix, sys.base_exec_prefix)" Fatal Python error: Py_Initialize: Unable to get the locale encoding ImportError: No module named 'encodings' Aborted (core dumped)
Trying after running "make altinstall" (which I had previously done for 3.4) is a bit more enlightening:
$ python3.4 -m venv --without-pip /tmp/issue22213-py34 $ /tmp/issue22213-py34/bin/python -c "import sys; print(sys.base_prefix, sys.base_exec_prefix)" /usr/local /usr/local $ PYTHONHOME=/usr/local /tmp/issue22213-py34/bin/python -c "import sys; print(sys.base_prefix, sys.base_exec_prefix)" /usr/local /usr/local $ PYTHONHOME=/tmp/issue22213-py34 /tmp/issue22213-py34/bin/python -c "import sys; print(sys.base_prefix, sys.base_exec_prefix)" Fatal Python error: Py_Initialize: Unable to get the locale encoding ImportError: No module named 'encodings' Aborted (core dumped) $ PYTHONHOME=/tmp/issue22213-py34:/usr/local /tmp/issue22213-py34/bin/python -c "import sys; print(sys.base_prefix, sys.base_exec_prefix)" Fatal Python error: Py_Initialize: Unable to get the locale encoding ImportError: No module named 'encodings' Aborted (core dumped) [ncoghlan@lancre py34]$ PYTHONHOME=/usr/local:/tmp/issue22213-py34/bin /tmp/issue22213-py34/bin/python -c "import sys; print(sys.base_prefix, sys.base_exec_prefix)" /usr/local /tmp/issue22213-py34/bin
I think what this is actually showing is that there's a fundamental conflict between mod_wsgi's expectation of being able to set PYTHONHOME to point to the virtual environment, and the way PEP-405 virtual environments actually work.
With PEP-405, all the operations in getpath.c expect to execute while pointing to the *base* environment: where the standard library lives. It is then up to site.py to later adjust the based prefix location, as can be demonstrated by the fact pyvenv.cfg isn't processed if processing the site module is disabled:
$ /tmp/issue22213-py34/bin/python -c "import sys; print(sys.prefix, sys.exec_prefix)" /tmp/issue22213-py34 /tmp/issue22213-py34 $ /tmp/issue22213-py34/bin/python -S -c "import sys; print(sys.prefix, sys.exec_prefix)" /usr/local /usr/local
At this point in time, there isn't an easy way for an embedding application to say "here's the standard library, here's the virtual environment with user packages" - it's necessary to just override the path calculations entirely.
Allowing that kind of more granular configuration is one of the design goals of PEP-432, so adding that as a dependency here.
grahamd commented
on Aug 23, 2014 grahamdmannequinMannequinAuthorIt is actually very easy for me to work around and I released a new mod_wsgi version today which works.
When I get a Python home option, instead of calling Py_SetPythonHome() with it, I append '/bin/python' to it and call Py_SetProgramName() instead.
Excellent! If I recall correctly, that works because we resolve the symlink when looking for the standard library, but not when looking for venv configuration file.
I also suspect this is all thoroughly broken on Windows - there are so many configuration operations and platform specific considerations that need to be accounted for in getpath.c these days that it has become close to incomprehensible :(
One of my main goals with PEP-432 is actually to make it possible to rewrite the path configuration code in a more maintainable way - my unofficial subtitle for that PEP is "getpath.c must die!" :)
- changed the title
[-]pyvenv style virtual environments unusable in an embedded system[/-][+]Make pyvenv style virtual environments easier to configure when embedding Python[/+]on Aug 24, 2014 grahamd commented
on Aug 24, 2014 grahamdmannequinMannequinAuthorI only make the change to Py_SetProgramName() on UNIX and not Windows. This is because back in mod_wsgi 1.0 I did actually used to use Py_SetProgramName() but it didn't seem to work in sane way on Windows so changed to Py_SetPythonHome() which worked on both Windows and UNIX. Latest versions of mod_wsgi haven't been updated yet to even build on Windows, so not caring about Windows right now.
That workaround would definitely deserve being wrapped in a higher-level API invokable by embedding applications, IMHO.
(Added Victor, Eric, and Steve to the nosy list here, as I'd actually forgotten about this until issue bpo-35706 reminded me)
Core of the problem: the embedding APIs don't currently offer a Windows-compatible way of setting up "use this base Python and this venv site-packages", and the way of getting it to work on other platforms is pretty obscure.
Victor may be thinking about it from time to time (or perhaps it's time to make the rest of the configuration changes plans concrete so we can all help out?), but I'd like to see this as either:
- a helper function to fill out the core config structure from a pyvenv.cfg file (rather than hiding it deeper as it currently is), or better yet,
- remove the dependency on all non-frozen imports at initialization and let embedders define Python code to do the initialization
In the latter case, the main python.exe also gets to define its behavior. So for the most part, we should be able to remove getpath[p].c and move it into the site module, then make that our Python initialization step.
This would also mean that if you are embedding Python but not allowing imports (e.g. as only a calculation engine), you don't have to do the dance of _denying_ all lookups, you simply don't initialize them.
But as far as I know, we don't have a concrete vision for "how will consumers embed Python in their apps" that can translate into work - we're still all individually pulling in slightly different directions. Sorting that out is most important - having someone willing to do the customer engagement work to define an actual set of requirements and desirables would be fantastic.
PEP-432 will allow to give with fine control on parameters used to initialize Python. Sadly, I failed to agree with Nick Coghlan and Eric Snow on the API. The current implementation (_PyCoreConfig and _PyMainInterpreterConfig) has some flaw (don't separate clearly the early initialization and Unicode-ready state, the interpreter contains main and core config whereas some options are duplicated in both configs, etc.).
See also bpo-35706.
57 remaining items
One needs to have the original (not venv) python in the
PATHenvironment variable to findpython313.dll.I'd strongly recommend using
AddDllDirectoryinstead (if you're not going to redistribute the runtime). Administrators can disable usingPATHfor DLL resolution, not to mention anyone can hijack your DLLs, so you have an extra variable to worry about when you rely on it.would it be safe to set
sys.executableafter the initialization to the current executable?I thought we had a configuration field for this already?
program_nameis just an input to our black box of calculating paths, but I'm pretty sure I would've exposed the real thing somewhere.If not, then yeah, it should be safe enough to just overwrite it. However, if someone just does
subprocess.run(["python", ...then Windows is going to choose apython.exein your application directory first. So if you have (the wrong) one there, it could still be a problem. But if all your code is usingsys.executablethen it'll be fine - it's just a variable.For the "emulating a venv" use case, I'd recommend setting
config.executable(which is then used to setsys.executable) over settingconfig.program_name.Setting
program_nameis for the case where the given name should NOT be invoked when launching Python subprocesses (usually because the embedding application is not itself a Python interpreter).Hmm, I think my last comment was misunderstood.
First of all, while in earlier versions of my software I used the "Windows embeddable ZIP" distribution, I ditched it now. It seemed too exotic. I only used it because venv environment didn't work well on Windows at that time. I am actually using venv in a normal way.The only exception to this is that my executable loads and initializes a custom 3rdParty Python module which needs a secret. And my executable has a different name so that it can easily be identified in the MS Windows task manager (or in
pson Linux).So, I am not "emulating" a venv.
I expect an existing Python installation including pip and venv.
I then use the venv module to create a virtual environment in .venv, use pip inside that virtual environment to install the required 3rdParty packages.
After configuration, to start my program, I use my custom executable instead of the python.exe from .venv\scripts.
While python.exe finds python313.dll automagically, my program needs it on the path.Regarding
AddDllDirectory, I tried adding a call toAddDllDirectory(...);with the path to my Python installation in my C main function before initializing python, but it doesn't even get there when that directory is not contained inPATH.Hmm... When venv creates the file scripts\python.exe in the .venv, does it do some DLL load magic?
Could the reason for this difference between python.exe and my own executable be related to this (from #112984 (comment))?
The venvlauncher.c is just launcher.c boiled down to just the venv parts, and none of the py.exe parts. The only change is that the executable embeds the expected Python executable name at compile time, as we can't just assume it'll be python[_d].exe anymore.
And it seems somewhat strange that the
venvlauncher.exe(which is renamed to python.exe if I understand correctly when the venv is created) needs to start a child process. This is different from the start of python in a normal installation (not in venv). Can you explain the reasoning behind this?For the "emulating a venv" use case, I'd recommend setting
config.executable(which is then used to setsys.executable) over settingconfig.program_name.I think I would disagree with this recommendation as
executableis undocumented as agetpathinput parameter. It is mostly used for internal testing.program_name, OTOH, is documented as one of the inputs. While we don't define the semantics for the Path calculation very clearly, I think it's safer to rely on that thanexecutable. Unless there's any unwanted side effect from settingprogram_name, it should be preferred to use it overexecutable.First of all, while in earlier versions of my software I used the "Windows embeddable ZIP" distribution, I ditched it now. It seemed too exotic. I only used it because venv environment didn't work well on Windows at that time. I am actually using venv in a normal way.
You say "in my software" but also "using venv in a normal way", so I'm not clear whether you are making an application that includes a Python runtime (for which the embeddable ZIP is intended), or just trying to avoid running the installer on your own machine?
While python.exe finds python313.dll automagically, my program needs it on the path.
There's nothing magical - they're in the same directory, and Windows uses that as the biggest hint to find a dependency.
Your executable either needs python313.dll in the same directory, or it needs its own launcher that doesn't directly depend on python313.dll, but is able to AddDllDirectory and then launch the executable that depends on it (the setting should be inherited). Alternatively, you could make the second executable a DLL and then LoadLibrary it, which will use the added DLL directory to resolve python313.dll and then you can call through your own interface. On second thought, this is probably the better approach.
Ultimately, Python is designed for a Unix-style system where all references are embedded absolute paths and nothing can be relocated. On Windows, the closest way to emulate this is to include your own copy of the runtime in your application directory (and this is what the embeddable distro is for).
It sounds like the best way to do what you seem to be doing (relying on someone else's install of Python) is to get them to provide the
python.exepath, launch that with your own script (potentially in a venv created by its own venv module) and your own extension modules that know how to talk back to your application. It's a bit of a workaround, but trust me, it'll make things much easier to test and much for stable for you and your users. Trying to directly load an unknown Python runtime that someone else is managing is not a great idea - it's very easy for circumstances to break it and render your application unusable.And it seems somewhat strange that the
venvlauncher.exe(which is renamed to python.exe if I understand correctly when the venv is created) needs to start a child process.Some ways to install Python don't allow directly loading its DLL, so launching a child process is the only way to do it. It would theoretically be possible to test this at runtime and only use the child process as a fallback (I have actually just been writing code that does this), but again, it's less reliable than using the proper interfaces.
We also get to take proper advantage of Windows preferring the application directory to ensure that we load the correct dependencies, or else we may crash at runtime. Previously, we either got lucky, or we had to copy practically the entire runtime into the venv to make it work reliably. With a small, self-contained launcher, we don't have to worry about either.
Out of interest, why do you find this strange? What were you expecting instead, and why?
whether you are making an application that includes a Python runtime (for which the embeddable ZIP is intended), or just trying to avoid running the installer on your own machine?
Neither nor.
After installing Python and creating a venv with
python -m venv venvand setting the VIRTUAL_ENV environment variable to thevenvdirectory, the only intended differences between running my executable and running thepython.exefromvenv\scriptsis that- my program needs to load a 3rdParty Python module (generated with SWIG) for which a license key is needed at runtime and the executable takes care of providing the license key
- and the executable has its own name such that it can be clearly identified in the task manager.
So, basically, my executable is "python.exe + a properly configured 3rd party module".
There's nothing magical - they're in the same directory, ...
No, they are not.
Thevenv\scriptsdirectory containspython.exe, but notpython313.dll.
And I can run thatpython.exedirectly, with a clean PATH:set PATH=%WINDIR%\System32;%WINDIR%;\%WINDIR%\System32\wbem C:\path\to\my_application\venv\Scripts>python.exeBy looking into the source code of
venvlauncher.c, I see that the magic happens there. As the comment at the top of the file says:This launcher looks for a nearby pyvenv.cfg to find the correct home directory, and then launches the original Python executable from it.
So the reason for the creation of the
python.exechild process ofpython.exeis exactly this DLL loading issue: To make sure that we find thepython*.dllin the same directory as the executable.My executable code does not use this logic, which is quite complex and requires very good knowledge of the Windows process creation API.
Duplicating this logic in my own code would be the worst solution.
Even if that "find python home (including DLL) from a virtual environment by parsing
pyvenv.cfg" would be available as a part of the initialization API, this would probably still not help in my use-case, because launching the python.exe from that python home would loose the initialization of the 3rdParty module.I think adding the Python home to the PATH is the most reasonable solution for my use-case.
Note that I control the PATH environment variable and use a clean PATH when I launch the executable.
From a security point of view, it is the same as howvenvwithpython.exeworks: An attacker would need write access to the python installation or the application installation.Probably my use-case is so special that I'll have to live with this little inconvenience.
Trying to directly load an unknown Python runtime that someone else is managing is not a great idea
I can't resist to say: This is exactly what venv is doing.
Trying to directly load an unknown Python runtime that someone else is managing is not a great idea
I can't resist to say: This is exactly what venv is doing.
We control both
venvand Python. If you happen to make a venv point to a Python that we didn't provide, you'll hit issues.My executable code does not use this logic, which is quite complex and requires very good knowledge of the Windows process creation API.
If it makes you feel any better, we've copy-pasted that code around more than a few times. It tries to handle many of the edge cases that people expect to Just Work, and mostly handles the most common ones (at least those presumed by POSIX developers, who expect it to work like
exec).- my program needs to load a 3rdParty Python module (generated with SWIG) for which a license key is needed at runtime and the executable takes care of providing the license key
I'm still intrigued by this. Not that I doubt it, but I've only come across one of these concepts before that couldn't be solved in some easier way.1 Can you provide any hint as to how this mechanism works? Or a link to docs, if it's documented somewhere? I'm interested.
But as you say:
Probably my use-case is so special that I'll have to live with this little inconvenience.
Yeah, I'm afraid so. Unless there's a sudden spate of similar irresolvable requests, I'm not keen to make the venv resolution a supported API here. (Supported API meaning anyone can call it, the behaviour is consistent across all platforms, safe across most [mis]configurations, and guaranteed not to change without a proper deprecation cycle. The internal behaviour of
venvlauncherandgetpathare not supported APIs in this sense - only the external behaviour of launchingScripts/python.exein an activated venv are guaranteed.)Footnotes
-
That one is OpenSSL, which expects to
GetProcAddress()on the running executable, which is fundamentally not a nice idea at all. But we haven't managed to get a patch contributed, and so we simply patch our own OpenSSL build to get the address from the_sslmodule instead. ↩
- added a commit that references this issue
on Jan 8, 2025 is there one example to load a virtual environment when embedding python? I created a venv and I'm trying to load it with python. I tried two things:
- Using Py_InitializeFromConfig and setting everything
- setting the venv path via PYTHONPATH env var.
This is the script im trying to run
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex, Settings from llama_index.llms.ollama import Ollama class AiContext: def __init__(self, model: str, url: str, data_directory: str): llm = Ollama(model = model, base_url = url) Settings.llm = llm documents = SimpleDirectoryReader(data_directory).load_data() index = VectorStoreIndex.from_documents(documents) self.query_engine = index.as_query_engine() def run_prompt(self, prompt: str) -> str: response = self.query_engine.query(prompt) return str(response)
without my venv set attempts,
llama_indexis obviously not found since its only present inside the venv. With 1 or 2, it findsllama_index, but I cannot useSimpleDirectoryReadernorOllama(no types that come from the packages inside the venv). When I run the same script inside the same venv, running activate and then justpython my_script.py, it works fine.I'm honestly quite confused, as there are no instructions on how to handle virtual environments and Python embedding. Is this a bug in Python, or is my way of setting up venv simply wrong? If so, is there a reference example that shows exactly what the right way is?
Not a solution actually, but maybe this helps others:
I gave up using the "Embeddable ZIP" distribution and switched to usingvenvwith a normal Python, but instead ofpython.exeI use a hand-crafted main program (also because I need a Windows and a Linux version, and the "Embeddable ZIP" is only available on Windows.I wanted a customized Windows executable with an icon, version information etc.
This was much easier, though there was still one quirk to work around:
It needs an environment variableVIRTUAL_ENVto find the python.exe in the venv.In my case I'm using a C library for PDF handling, for which I generated a Python wrapper with SWIG.
Note: This is for Windows only. For Linux, some modifications are necessary.
My executable behaves like python.exe, because it calls
Py_RunMain, but I think this could be changed quite easily.#include <windows.h> #include "Python.h" #include <string.h> #include <stdlib.h> #include <stdio.h> #include ... #define PATH_MAX 200 PyObject* PyInit__pdf(void); /* We need this to workaround a quirk when using venv in combination with anything else but a (possibly renamed) python.exe */ void get_interpreter_path(const char* virtual_env, wchar_t* interpreter_path) { char python_exe[PATH_MAX]; sprintf(python_exe, "%s\\scripts\\python.exe", virtual_env); wchar_t* temp = Py_DecodeLocale(python_exe, NULL); wcsncpy(interpreter_path, temp, PATH_MAX); if (interpreter_path[PATH_MAX - 1] != 0) { printf("Interpreter path too long!"); exit(1); } } void init_python( int argc, char* argv[], PyStatus* status, PyPreConfig* preconfig, PyConfig* config ) { int rc; static wchar_t interpreter_path[PATH_MAX]; PyPreConfig_InitPythonConfig(preconfig); preconfig->utf8_mode = 1; preconfig->allocator = PYMEM_ALLOCATOR_MALLOC; *status = Py_PreInitialize(preconfig); if (PyStatus_Exception(*status)) { goto pre_exception; } PyConfig_InitPythonConfig(config); PyConfig_SetBytesArgv(config, argc, argv); if (PyStatus_Exception(*status)) { goto init_exception; } const char* virtual_env = getenv("VIRTUAL_ENV"); if (virtual_env != NULL) { get_interpreter_path(virtual_env, interpreter_path); PyConfig_SetString(config, &(config->program_name), interpreter_path); } // provide the PDF library as a Python module rc = PyImport_AppendInittab("...", PyInit__pdf); if (rc != 0) { printf("Returncode from PyImport_AppendInittab: %d\n", rc); PyStatus_Error("Unable to initialize PDF library"); goto pre_exception; } *status = Py_InitializeFromConfig(config); if (PyStatus_Exception(*status)) { goto init_exception; } PyConfig_Clear(config); return; init_exception: PyConfig_Clear(config); pre_exception: Py_ExitStatusException(*status); } int main( int argc, char* argv[] ) { int rc = 0; PyStatus status; PyPreConfig preconfig; PyConfig config; /* Initialize the PDF library at the C level. */ ... /* Start Python normally and provide the PDF library to Python */ init_python(argc, argv, &status, &preconfig, &config); rc = Py_RunMain(); /* Shutdown the PDF library */ ... PyConfig_Clear(&config); if (PyStatus_Exception(status)) { /* Display the error message and exit the process with non-zero exit code */ Py_ExitStatusException(status); } else if (PyStatus_IsExit(status)) { return status.exitcode; } return rc; }
Reacted by Steve DowerNot a solution actually, but maybe this helps others
This is our preferred solution, and your users will have a much better time when you embed Python properly (like this) rather than trying to hack around the interactive REPL (like everyone else wants to).
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
Linked PRs