Repository navigation
Split up Modules/gcmodule.c #113688
Description
Activity
A single file with both versions of the GC sounds quite confusing so I favor splitting it. Your scheme looks okay to me. Possibly we need an internal header file that's shared between the .c files. The original gcmodule.c was written with the assumption that small functions could be inlined by the compiler as it wishes. With split files, an LTO build probably results in the same inlining. Otherwise, we might need some inline functions in the .h file. As with any optimization, it's probably best to do that only as required for payoff.
I’m fine with the splitting as well as I really don’t think we should have both versions in the same file so let’s go ahead unless someone has some strong reservations.
Refarding the performance concer a, we do have benchmarks that are checking the performance of GC runs in the pyperformance suite so we can always monitor if changes we made are significant. I am mentioning this so we are not too conservative with the changes and being worried that they are going to affect performance a lot because at the end of the day is very difficult to actually predict how these kind of alterations are going to affect performance.
So my proposal is that we do optimise for maintainability and then check performance and adapt from there.
Reacted by Gregory P. Smith, Petr Viktorin and Erlend E. AaslandAgreed, I think this split makes sense. What y'all have already said above.
I agree, too. Other runtime parts are still possible to maintain implementation details except caring whether GIL exists or not. (or with a small cost)
But GC is wholly different. GC should care where objects are located. For example, some objects existed in the interpreter area only, but now we should move them into the thread area. Those kinds of things require separation, and this suggestion will improve maintainability.
And if we introduce more GC implementation details(STW, deferred refcount) in a single file, I can not imagine what will happen :)But GC is wholly different. GC should care where objects are locate
Technically, the current one doesn't care, and that was an advantage. The new one does, which is a different advantage ;)
Reacted by Donghee NaGreat - it seems like there's consensus on splitting the files.
@nascheme, I was thinking that maybe
Python/gc.ccould#includeone ofPython/gc_gil.corPython/gc_free_threaded.c. Something like:// in Python/gc.c #ifdef Py_GIL_DISABLED # include "gc_free_threaded.c" #else # include "gc_gil.c" #endif
We already do this in a few places, with clinic generated files, in
Modules/socketmodule.c, obmalloc with mimalloc files, and a few other modules.It's a bit icky, but it avoids both the concerns around inlining and complexity with configuring the build system to include one or the other file since that's handled at the macro pre-processor level.
Reacted by Erlend E. AaslandI'd prefer not to do that include of .c files. It might make debugging harder, for example. I'd include both files in the build always and then put
ifdefsinside of them to disable code if needed. That avoids needing extra logic when generating the makefile or whatever.This is something we should be able to adjust the how of based on practical experiences if the initial approach winds up causing any issues. Debuggers know which file any given line of code came from, I'm not worried at all about that for #includes.
Reacted by Erlend E. Aasland- added a commit that references this issue
on Jan 5, 2024 - added a commit that references this issue
on Jan 8, 2024 This change broke
--with-dtracebuild on Solaris (or other systems where object files need to be modified with dtrace before linking) with error like this:Undefined first referenced symbol in file __dtraceenabled_python___gc__start Python/gc.o __dtraceenabled_python___gc__done Python/gc.o __dtrace_python___gc__start Python/gc.o __dtrace_python___gc__done Python/gc.o ld: fatal: symbol referencing errors collect2: error: ld returned 1 exit status#113814 fixes the issue.
- added a commit that references this issue
on Jan 8, 2024
Feature or enhancement
The free-threaded builds require substantial changes to the garbage collection implementation, but we don't want to make those changes to the default build. In other parts of CPython, we have used "local"
#ifdef Py_GIL_DISABLEDguards around free-threaded specific bits, but I think doing this throughoutModules/gcmodule.cwill make the GC harder to maintain than having separate files for the free-threaded and default builds. Additionally, I think @markshannon already suggested splitting the core GC parts from the Python "gc" module.There's definitely an increased maintenance cost for having two GC implementations in separate files, but I think this maintenance will be less than having a single file with intermixed implementations and lots of
#ifdef Py_GIL_DISABLEDguards.Here's a proposed split:
Modules/gcmodule.c-gcPython package interfacePython/gc.c- core GC functionality common to both build configurations (GIL & free-threaded)Python/gc_gil.c- GIL (non-free-threaded) specific implementationsPython/gc_free_threaded.c- free-threaded specific implementationsAt first, I'd expect most code to be in either
Modules/gcmodule.corPython/gc.c. As work on the free-threaded build continues, I would expectPython/gc.cto shrink as code is moved intoPython/gc_gil.candPython/gc_free_threaded.c.@nascheme @markshannon @pablogsal @DinoV @vstinner
Linked PRs