Repository navigation
Creating an ungodly amount of sub interpreters in a short amount of time causes memory debug assertions. #123134
Description
Activity
Can you reproduce this behaviour on Linux? or MacOS? Or what happens if you sleep a bit before spawning the sub-interpreters? I'm not an expert in threads but I'd say that interpreters are spawning too fast for the OS to follow up =/ Can you check whether the same occurs with HEAD instead of 3.12.5? (maybe there are some bugfixes that are not yet available)
At a glance, this could possibly be related to a memory allocation failure or something similar.
auto globals = PyDict_New(); auto code = Py_CompileString(text.data(), __FILE__, Py_eval_input); auto result = PyEval_EvalCode(code, globals, globals);
These lines need a
NULLcheck if an error occurred, otherwise there will be aNULLpointer dereference in the call toPy_DecRef.At a glance, this could possibly be related to a memory allocation failure or something similar.
auto globals = PyDict_New(); auto code = Py_CompileString(text.data(), __FILE__, Py_eval_input); auto result = PyEval_EvalCode(code, globals, globals);
These lines need a
NULLcheck if an error occurred, otherwise there will be aNULLpointer dereference in the call toPy_DecRef.Py_DecRefis the function version ofPy_XDECREF. Though I can see the confusion betweenPy_DecRefandPy_DECREF. If there was anullptr, we'd have a different set of errors, and the debug memory assertions wouldn't be complaining that all memory was not written with theFORBIDDENBYTEvalue.Can you reproduce this behaviour on Linux? or MacOS?
I will work on getting the Linux/macOS behaviors checked when I can. My WSL2 environment is woefully out of date 😅. macOS might be more difficult, as I'm unaware of how the_dversion of thePython.frameworkmight be linked against.Or what happens if you sleep a bit before spawning the sub-interpreters?
Sleeping might fix it, but might also hide the issue. The goal here was just to run a bunch of interpreters, not sleep for the sake of working around possible synchronization issues
I'm not an expert in threads but I'd say that interpreters are spawning too fast for the OS to follow up =/
This is not how threads or operating systems work. There is a synchronization issue or an ABA problem with the debug memory assertions in the C API's internals
Can you check whether the same occurs with HEAD instead of 3.12.5? (maybe there are some bugfixes that are not yet available)
I will also work on giving this a go.
Py_DecRefis the function version ofPy_XDECREFOh! TIL. I still would suggest adding a
NULLcheck -- an error indicator could be set, which would break subsequent calls toPyErr_Occurred.The error is outside of the Python C API as one might usually interact with it. i.e., only on subinterpreter initialization (i.e., creating a
PyThreadState) or on subinterpreter destruction (Py_EndInterpreter) does the error occur (typically the latter).PyErr_Occurredis not being called anywhere in the MRE, nor in the actual traceback for the failure.Right, but it could cause memory corruption somewhere, and cause the error somewhere else down the line. Regardless, I wasn't able to reproduce this on Linux, so this might be Windows-specific.
@picnixz I can confirm the behavior still exists on Windows on
HEAD. Same errors can occurRight, but it could cause memory corruption somewhere, and cause the error somewhere else down the line. Regardless, I wasn't able to reproduce this on Linux, so this might be Windows-specific.
PyErr_Occurredonly performs a read on the current thread state object. If an error is present when it shouldn't be that still points to a memory corruption error occurring between the destruction and initialization of aPyThreadState.Pleased to back up @ZeroIntensity and say that this appears to be a Windows specific issue. I was unable to replicate this memory assertion error on Linux.
However, when executing this code under TSan, I got the following data race error that seems to indicate the same kind of memory corruption COULD be taking place.
WARNING: ThreadSanitizer: data race (pid=11259) Write of size 8 at 0x7f24e35e16c0 by thread T10: #0 qsort_r <null> (463-interpreters+0x9bc2e) (BuildId: 4e94833395fedac62b3c4caa6921ff713e25cff1) #1 qsort <null> (463-interpreters+0x9bec7) (BuildId: 4e94833395fedac62b3c4caa6921ff713e25cff1) #2 setup_confname_table /usr/src/python3.12-3.12.3-1ubuntu0.1/build-shared/../Modules/posixmodule.c:13543:5 (libpython3.12.so.1.0+0x3b6946) (BuildId: e2e0fbc52bf8cb7daf99595e16454e44217040f4) #3 setup_confname_tables /usr/src/python3.12-3.12.3-1ubuntu0.1/build-shared/../Modules/posixmodule.c:13572:9 (libpython3.12.so.1.0+0x3b6946) #4 posixmodule_exec /usr/src/python3.12-3.12.3-1ubuntu0.1/build-shared/../Modules/posixmodule.c:16905:9 (libpython3.12.so.1.0+0x3b6946) ... Previous write of size 8 at 0x7f24e35e16c0 by thread T18: #0 qsort_r <null> (463-interpreters+0x9bc2e) (BuildId: 4e94833395fedac62b3c4caa6921ff713e25cff1) #1 qsort <null> (463-interpreters+0x9bec7) (BuildId: 4e94833395fedac62b3c4caa6921ff713e25cff1) #2 setup_confname_table /usr/src/python3.12-3.12.3-1ubuntu0.1/build-shared/../Modules/posixmodule.c:13543:5 (libpython3.12.so.1.0+0x3b6946) (BuildId: e2e0fbc52bf8cb7daf99595e16454e44217040f4) #3 setup_confname_tables /usr/src/python3.12-3.12.3-1ubuntu0.1/build-shared/../Modules/posixmodule.c:13572:9 (libpython3.12.so.1.0+0x3b6946) #4 posixmodule_exec /usr/src/python3.12-3.12.3-1ubuntu0.1/build-shared/../Modules/posixmodule.c:16905:9 (libpython3.12.so.1.0+0x3b6946) Location is global '??' at 0x7f24e2e85000 (libpython3.12.so.1.0+0x75c880)
This only happened 3 times, and around the 42 thread generation step on average.
However, when executing this code under TSan, I got the following data race error that seems to indicate the same kind of memory corruption COULD be taking place.
A quick look suggests this TSan failure is a local data-race to posixmodule.c: There appears to be no protection from multiple threads importing posixmodule simultaneously, and hence multiple threads trying to simultaneously qsort in-place one of the three static tables.
This module is used on Windows, but I'm not sure if any of the macros enabling those
qsortcalls (HAVE_FPATHCONF,HAVE_PATHCONF,HAVE_CONFSTR, orHAVE_SYSCONF) are enabled in the Windows build. My guess would be no...I don't know if overlapping qsorts on the same array will actually corrupt memory, but I doubt there's any guarantee it won't. (
qsort_rdoesn't help, that makes the comparison function thread-safe, not the array being sorted, according to the manpage.)So probably a separate issue, and a more-focused/reliable MRE could be made for it.
67 remaining items
I'm going to focus on the allocator issue for the time being, there's a lot of uncharted water in this, I wouldn't be surprised if like, 4 new issues are needed 😅
(If you would like to author PRs for docs change and whatnot, that would be great! You've clearly put much more time into this issue than I have.)
Ok, regarding the allocation issue, I'm worried about breaking code here, because theoretically someone could be relying on all threads being affected by modifying the
Rawdomain. Speculatively, we could modify what allocatorPyMem_Raw*reaches for depending on the current thread state -- for example, if it'sNULL, then it's not safe to try and call whatever Python has registered, and should just try and go formalloc. But even that could break code (think if someone allocates something without the GIL, such as in aPy_BEGIN_ALLOW_THREADSblock, and then tries to free that memory after the GIL has been re-acquired), and wouldn't play well with debuggers.Reacted by Paul "TBBle" HampsonI guess a fix could be to try and make the uses of
_PyMem_SetDefaultAllocatorthread-local somehow, but that's going to be a PITA.Reacted by Paul "TBBle" HampsonI couldn't think of any way to make
_PyMem_SetDefaultAllocatorgenerally useable; I suspect our best bet would be to make it assert that it's only called at safe times: Before/during initialisation and finalisation of the main interpreter. And we can also note thatPyMem_Raw*is not threadsafe during those times.Although that's both pretty messy, somewhat complicated, and may break existing non-crashing code.
There's probably a similar reproduction case (in release builds only) even without the
_PyMem_SetDefaultAllocatorissue:(UNTESTED)
#include <thread> #include <Python.h> int main() { auto pre_config = PyPreConfig {}; PyPreConfig_InitIsolatedConfig(&pre_config); if (auto status = Py_PreInitialize(&pre_config); PyStatus_Exception(status)) { Py_ExitStatusException(status); } auto a = PyMem_RawMalloc(100); PyMem_SetupDebugHooks(); PyMem_RawFree(a); return 0; }
This is currently valid: As I recall the only valid place to call
PyMem_SetupDebugHooks()(orPyMem_SetAllocator()when not a trivial wrapper for existing allocator) is betweenPy_PreInitializeandPy_Initialize.However, this will reproduce the same bug, as
PyMem_SetupDebugHooks()will change the allocate\free function pointer and a block allocated under the normal allocator will be free'd by the debug allocator.So we can possibly fix both holes by noting that
PyMem_Raw*is not thread-safe before the main interpreter'sPy_InitializeFromConfig, and must not be called before any use ofPyMem_SetAllocatorwhich does not meet the stronger version ofPyMem_SetAllocator's contract I proposed in the second paragraph of #123134 (comment):any newly-set allocator after init needs to be able to free allocations created by the allocator it wraps
We would also need to extend the case of "Python has finish initializing" to also include "or PyMem_Raw* APIs have been called".
PyMem_SetupDebugHooks()does not currently meet this requirement, so it must not be called afterPyMem_RawMallochas been called, making this repro case invalid.A stronger change that closes this second hole would be to deprecate-remove
PyMem_SetupDebugHooks()in favour of the existingPyPreConfigparameter (like we did forPy_SetProgramName), plus a public API that can wrap a given allocator with debug hooks and the inverse unwrapping API, which could also be used to fix the small race condition in_PyMem_SetDefaultAllocator, i.e. the "new wrinkle".Speaking as a user here: I've given up on the
subinterpreter API-- for my specific use case, which involves spawning ~8 python threads for parallel processing of a bioinformatics workload (the solution is embedded in Go) -- subinterpreters weren't ready at all.- I couldn't get them to work except when importing a pthreads library in Go (I and another engineer couldn't get them to work inside Go goroutines)
- Even after all the Cgo/pthreads magic in Go the solution was 2 times slower than spawning separate Python processes and communicating with them via stdin/stdout
- I couldn't find a way to finalise the main python interpreter that wouldn't segfault. I decided this wasn't good enough for production.
Obviously these were unchartered waters, but my reading of the subinterpreter API was that this was ready for multithreaded use. It's perhaps worth updating the documentation:
Using Py_NewInterpreterFromConfig() you can create a sub-interpreter that is completely isolated from other interpreters, including having its own GIL. The most important benefit of this isolation is that such an interpreter can execute Python code without being blocked by other interpreters or blocking any others. Thus a single Python process can truly take advantage of multiple CPU cores when running Python code.
I couldn't think of any way to make
_PyMem_SetDefaultAllocatorgenerally useable; I suspect our best bet would be to make it assert that it's only called at safe times: Before/during initialisation and finalisation of the main interpreter. And we can also note thatPyMem_Raw*is not threadsafe during those times.My idea was to make internal uses of
_PyMem_SetDefaultAllocatoronly affect the current thread, so functions that need to use the raw allocator wouldn't be breaking thread safety. But if there are issues withPyMem_SetupDebugHooksas well, then that wouldn't work. I was able to confirm your reproducer withPyMem_SetupDebugHooks.A deprecation in favor of a config option is probably better, and then I guess updating thread safety notes about
PyMem_Raw*is the best way to go.Obviously these were unchartered waters, but my reading of the subinterpreter API was that this was ready for multithreaded use. It's perhaps worth updating the documentation:
It is ready, in the sense that we have all the necessary subinterpreter isolation ready, but my understanding of the problem is that we sort of don't have public APIs to really take advantage of it yet. I think my solution with manually attaching a thread state for the main interpreter and then switching to the subinterpreter was the valid way to do it--it's just not very pretty.
Specifically, which parts of subinterpreters "did not work", and what needs to be documented better? As far as I can see, most of the people that know about subinterpreters are the ones working on them, so it would be good to get some feedback from users on what needs to be made clearer.
Reacted by Paul "TBBle" HampsonI posted a C-only version of my pthreads code earlier on in this thread, but it must have been deleted (the thread got quite unwieldy so maybe somebody tried to clean it up and deleted that post). I wrote the code using the C API documentation as a starting point I believe @tbbie was commenting on it.
My personal question would have been: is that code correct?
I think having a pthreads template (which is effectively a multithreading standard for C) with works.
I've tried to dig out that code and it's something like this (the idea is run a tight loop from each subinterpreter to test for potential segfaults, etc...). I believe that in this particular instance I was able to call Py_Finalise (I wasn't able to call that in Cgo without segfault):
#ifndef SUBINTERPRETERS_C #define SUBINTERPRETERS_C #include <Python.h> #include <stdio.h> #include <stdlib.h> #include <pthread.h> // Define the number of threads to create #define NUM_THREADS 4 // Define the upper limit for the range in our Python calculation #define RANGE_LIMIT 10000000 // Configure the Python interpreter PyInterpreterConfig config = { .check_multi_interp_extensions = 1, // Check for multi-interpreter compatible extensions .gil = PyInterpreterConfig_OWN_GIL, // Each interpreter gets its own GIL }; // Define a struct to hold the result of each thread's execution typedef struct { int error_code; char error_message[256]; } ThreadResult; // Declare a global mutex to protect interpreter creation //pthread_mutex_t interpreter_mutex = PTHREAD_MUTEX_INITIALIZER; // Function that each thread will run void *run_python_code(void *arg) { // Cast the argument to ThreadResult pointer ThreadResult *result = (ThreadResult *)arg; // Initialize the result result->error_code = 0; result->error_message[0] = '\0'; PyThreadState *tstate = NULL; // Lock the mutex before creating a new interpreter //pthread_mutex_lock(&interpreter_mutex); // Create a new Python interpreter PyStatus status = Py_NewInterpreterFromConfig(&tstate, &config); // Unlock the mutex after creating the interpreter //pthread_mutex_unlock(&interpreter_mutex); // Check if interpreter creation was successful if (PyStatus_Exception(status)) { result->error_code = 1; snprintf(result->error_message, sizeof(result->error_message), "Couldn't spawn new interpreter"); return NULL; } // Declare Python objects we'll use PyObject *pLocal, *pSum, *pRange, *pIter, *pItem; // Create a new dictionary for local variables pLocal = PyDict_New(); // Create a Python long object initialized to 0 pSum = PyLong_FromLong(0); // Add 'sum' to the local dictionary PyDict_SetItemString(pLocal, "sum", pSum); // Create a range object pRange = PyObject_CallFunction(PyObject_GetAttrString(PyImport_AddModule("builtins"), "range"), "l", RANGE_LIMIT); // Get an iterator from the range pIter = PyObject_GetIter(pRange); // Iterate over the range while ((pItem = PyIter_Next(pIter))) { // Add the current item to the sum PyObject *pNewSum = PyNumber_Add(pSum, pItem); Py_DECREF(pSum); pSum = pNewSum; Py_DECREF(pItem); } // Print the final sum PyObject_Print(pSum, stdout, 0); printf("\n"); // Clean up Python objects Py_DECREF(pLocal); Py_DECREF(pSum); Py_DECREF(pRange); Py_DECREF(pIter); // End the interpreter Py_EndInterpreter(tstate); return NULL; } int main() { // Initialize the Python interpreter Py_Initialize(); // Declare arrays to hold thread IDs and results pthread_t threads[NUM_THREADS]; ThreadResult results[NUM_THREADS]; int num_threads_created = 0; // Create threads for (int i = 0; i < NUM_THREADS; i++) { int ret = pthread_create(&threads[i], NULL, run_python_code, &results[i]); if (ret) { fprintf(stderr, "Error creating thread %d: %d\n", i, ret); break; } else { printf("Successfully created thread %d:\n", i); } num_threads_created++; } // Wait for all created threads to finish for (int i = 0; i < num_threads_created; i++) { pthread_join(threads[i], NULL); // Print any errors that occurred in the thread if (results[i].error_code) { fprintf(stderr, "Thread %d error: %s\n", i, results[i].error_message); } printf("Thread %d completed with error code: %d\n", i, results[i].error_code); } // Destroy the mutex //pthread_mutex_destroy(&interpreter_mutex); // Finalize the Python interpreter Py_Finalize(); return 0; } #endifAt a glance, the bug there is calling
Py_NewInterpreterFromConfigwithout the GIL. See what I did earlier in the thread withPyThreadState_Newto get around that.As I said earlier, I've essentially moved on from this at this stage, and I'm using OS-level processes to manage the interaction between my Go and Python code, and it's been 2x faster than working with subinterpreters and much easier to work with. Personally I'm happy to leave it at that.
But again, speaking from the user perspective, I think having a working example in the documentation would have been helpful at the time (and by this I mean a full example: starting the threads, doing calculation and cleaning up, holding the GIL correctly, etc...). And I think the example should use the most standard multithreading library, like pthreads. I realise that the example will be quite long, and perhaps it will need to be linked from the documentation, but I think as it stands, it's essentially impossible to just read the documentation and come up with a correct example of multithreaded subinterpreter code.
Reacted by Peter BiermaI posted a C-only version of my pthreads code earlier on in this thread, but it must have been deleted (the thread got quite unwieldy so maybe somebody tried to clean it up and deleted that post). I wrote the code using the C API documentation as a starting point I believe @tbbie was commenting on it.
GitHub folds the middle of long issues, clicking this link should show it.
What appears to be the necessary dance for holding the GIL to create an interpreter on the new thread is shown in #123134 (comment), everything from
auto tstate = PyThreadState_New(interp);toPyThreadState_Delete(tstate);. The first three lines are what the threading module'sThreadHandle_Startandthread_rundo, the rest of the stuff afterPy_NewInterpreterFromConfigis basically to clean up the temporary thread state becausePy_NewInterpreterFromConfighas created a new one for our separate-GIL interpreter. We've approximately open-codedPyGILState_Ensurefor an explicit interpreter (the default is the main interpreter anyway) and andPyGILState_Releasefor an explicit thread state, but without a bunch of safety rails for nested calls.So I believe something like this would work, but is not supported:
auto gilstate = PyGILState_Ensure(); assert(gilstate == PyGILState_UNLOCKED); auto main_interpreter_state = PyThreadState_GetCurrent(); if (auto status = Py_NewInterpreterFromConfig(&state, &config); PyStatus_IsError(status)) { std::println("Failed to initialize thread state {}", count); // NOTE: Need to clean up the thread state we created earlier. // See https://github.com/python/cpython/blob/main/Python/crossinterp.c#L1833-L1841 return; } // Swap back to the old thread state, clear it, and then switch back to our new state. auto new_thread_state = PyThreadState_Swap(main_interpreter_state); assert(new_thread_state == state); PyGILState_Release(gilstate); auto temp_thread_state = PyThreadState_Swap(state); assert(temp_thread_state == NULL); // Do stuff with your new interpreter
I've just realised
Py_EndInterpreterprobably doesn't clean up the thread state, we probably should end with something like this, butPyThreadState_Cleardepends on holding the GIL, so does that need to be done beforePy_EndInterpreter? (Or per below, maybe that cleanup is done byPy_EndInterpreteralready...)
Exploring more:
As it happens, the new module for exposing interpreter creation to Python drops the GIL before calling
Py_NewInterpreterFromConfigso maybe holding the GIL or even having an active thread state is not necessary forPy_NewInterpreterFromConfigafter all? This code-flow also throws away the new interpreter's thread state (except in a unit test, which is actually quite close to our use-case here), and then creates a new thread state on-demand for executing code.Anyway, as I just linked, there's a smaller and safer C API prefixed with
_PyXI_which wraps up a bunch of this stuff but with a more-structured approach (and not public, I think; hopefully it will become so in future), and that's then exposed to Python code by the in-tree_interpretersmodule, and that's wrapped by theinterpreters-PEP-734module on pypi. In this new API,_PyXI_Enterand_PyXI_Exitare roughly analogous toPyGILState_EnsureandPyGILState_Releasebut for interpreters, and simplify a bunch of this stuff.I also note that
_PyXI_EndInterpreteralso does not clear up a temporary thread state used forPy_EndInterpreter(nor does the open-coded version in the unit tests), so maybePy_EndInterpreterdoes already clear up the thread state for the interpreter, and I just didn't notice the code to do that.Yes,
_PyXI*are Eric's magic functions that he's using to develop PEP-734, I think. We can't use them yet :)As it happens, the new module for exposing interpreter creation to Python drops the GIL before calling Py_NewInterpreterFromConfig so maybe holding the GIL or even having an active thread state is not necessary for Py_NewInterpreterFromConfig after all?
It might be a bug. It's all private, so I doubt it's had the kind of stress testing like what we're doing here. Crossinterpreter stuff is all undocumented at the moment. If you read the discussion in my PR about the GIL and subinterpreters, the behavior that you see right now isn't necessarily intended, because it's untested and support is explicitly disallowed.
If you read the discussion in my PR about the GIL and subinterpreters, the behavior that you see right now isn't necessarily intended, because it's untested and support is explicitly disallowed.
I was only noting that if one puts up the code we got working here side-by-side with the implementation of
PyGILState_EnsureandPyGILState_Release(and with the threading module) one can see that they're all doing the same operations, as a reassurance that we've ended up with the right way of getting a GIL established in a new thread and cleaning up afterwards. That's not really specific to sub-interpreters, it just happens that we create a sub-interpreter between the two calls, and then switch back to the thread state we had before that happened.I'm aware that
PyGILState_*is not supported in combination with sub-interpreters. The reason I described_PyXI_Enteras "PyGILState_Ensurefor interpreters" is because it does exactly the same thing asPyGILState_Ensureexcept it takes an interpreter state as an argument, whilePyGILState_Ensureuses the runtime's "gilstate.autoInterpreterState" pointer, which is always the main interpreter.It might be a bug. It's all private, so I doubt it's had the kind of stress testing like what we're doing here.
To the best of my knowledge, at no point did taking the GIL or not before
Py_NewInterpreterFromConfigaffect the results of any repro case we've had here. So the stress testing we've done indicates that it's not required, even though the docs ofPy_NewInterpreterFromConfigstate that it is. (They also at one point say it's not required, but the overall vibe seems to be that it is formally required, so that's probably an oversight or an abstraction-break.)That's also what I understood from reading the function, since before the point where it drops the GIL internally anyway, it does nothing but GIL-unneeded actions as far as I recall. The problems we saw inside
Py_NewInterpreterFromConfigin 3.12 occur later in the function, when the new GIL has already been established (or the old GIL retaken), and are not related to the act of interpreter creation itself.Anyway, it may become technically required in the future, and an API that is formally more restricted than its implementation is the correct way 'round IMHO.
I also don't want future discussions of this issue to get bogged down in unrelated API discussions, so being formally correct in repro cases is desirable.
Reacted by Peter BiermaOk, @TBBle, I think it's time to get some PRs down. I don't know if there's anything that can be done about the
PyMem_SetupDebugHooksproblem, but we can fix the issues with_PyMem_SetDefaultAllocatorinternally by making it only affect the current thread, or possibly switch the internal uses that need the true systemmallocover to_PyMem_RawMalloc, as that can't be hooked. Would you like to deal with updating the documentation for the allocator contract?I'm pretty sure that #130287 will fix this.
Reacted by Paul "TBBle" Hampson
Bug report
Bug description:
Hello. While working on a small joke program, I found a possible memory corruption issue (it could also be a threading issue?) when using the Python C API in a debug only build to quickly create, execute python code, and then destroy 463 sub interpreters. Before I post the code sample and the debug output I'm using a somewhat unique build environment for a Windows developer.
When running the code sample I've attached at the bottom of this post, I am unable to get the exact same output each time, though the traceback does fire in the same location (Due to the size of the traceback I've not attached it, as it's about 10 MB of text for each thread). Additionally, I sometimes have to run the executable several times to get the error to occur. Lastly, release builds do not exhibit any thread crashes or issues as the debug assertions never fire or execute.
The error output seems to also halt in some cases, either because of stack failure or some other issue I was unable to determine and seemed to possibly be outside the scope of the issue presented here. I have the entire error output from one execution where I was able to save the output.
The output cut off after this, as the entire program crashed, taking my terminal with it 😅
You'll find the MRE code below. I've also added a minimal version of CMakeLists.txt file I used so anyone can recreate the build with the code below (Any warnings, or additional settings I have do not affect whether the error occurs or not). The code appears to breaks inside of
_PyObject_DebugDumpAddress, based on what debugging I was able to do with WinDbg.Important
std::jthreadcalls.join()on destruction, so all threads auto-join once thestd::vectorgoes out of scope.Additionally this code exhibits the same behavior regardless of whether it is a
thread_localor declared within the lambda passed tostd::threadmain.cxx
CMakeLists.txt
Command to build + run
CPython versions tested on:
3.12
Operating systems tested on:
Windows