Repository navigation
Disable importing legacy single-phase init extensions within subinterpreters in --disable-gil build #117649
Description
Activity
Note that this only relates to legacy subinterpreters (not isolated, created by
Py_NewInterpreter()). With isolated interpreters single-phase init modules already triggerImportError.Raising
ImportErrorunder--disable-gilseems like a reasonable short-term solution.Reacted by Sam Gross- changed the title
[-]Disable importing single-phase init extensions in subinterpreters in `--disable-gil` build[/-][+]Disable importing legacy single-phase init extensions within subinterpreters in `--disable-gil` build[/+]on Apr 8, 2024 You might want use the current mechanism: require
PyInterpreterConfig.check_multi_interp_extensionsto be 1 forPy_NewInterpreterFromConfig, and set it to 1 forPy_NewInterpreter(i.e._PyInterpreterConfig_LEGACY_INIT).
And perhaps disable_imp._override_multi_interp_extensions_check.Note that there are two ways to implement the limitation:
- allow the import only in the main interpreter
- allow the first import, regardless of what interp it's made in
The first was easier to implement; we might want to switch to the second later. If/when we do,
--disable-gilbuilds might want to follow along.Reacted by Eric Snow- added 4 commits that reference this issue
on Apr 11, 2024
Feature or enhancement
When importing a single-phase init extension in a subinterpreter, Python will make a shallow copy of the module's dictionary, which can share (non-immortal) objects between interpreters.
This does not work properly in the
--disable-gilbuild, and we should disable it by raising anImportError, at least for now. We can investigate how to support this in the future.There are currently some unit tests for this case. Those tests pass, but that's mostly because they are simple and small changes to things like the GC will cause them to crash.
The underlying problems are not directly related to the GIL, but rather because the GC and our mimalloc integration in the
--disable-gilbuild assume that non-immortal objects are isolated by interpreter:tp_traverseare also reachable from the per-interpreter mimalloc heaps. Violating this assumption can cause flags to be set to an inconsistent state.Linked PRs